One of the challenges for a global organization like Air Liquide is the diverse collection of distributed control systems (DCS) operating our air separation units (ASU) and associated equipment around the world. However, since many of the company’s plants operate similarly, we also have the opportunity to leverage a common library of unit modules to provide consistency of execution and a platform for advanced functionality such as modular procedural automation (MPA).
MPA promises to help automate start-up, shutdown and other data-driven procedures that can increase efficiency, optimize production, and reduce wear and tear on equipment—all while helping operators be more productive and act more confidently. But the diversity of legacy systems currently installed requires that we use a supervisory controller that can be retrofitted onto existing systems and seamlessly integrated with newer control systems as facilities are upgraded. Further, since the ASUs directly feed pipelines or our clients’ production units, unscheduled downtime to facilitate programming changes or code updates to the supervisory controller is not an option. An agnostic solution that would accommodate the company’s requirements and challenges was needed.
Synchronize with caution
The first implementation of our process unit modular automation (PUMA) solution was deployed at Air Liquide’s Ingleside, Texas, ASU facility. It consists of an Emerson DeltaV PK controller, which acts as the supervisory platform for MPA, connected with a legacy Foxboro DCS through a Cogent DataHub (Figure 1). The DataHub effectively bridges the older OPC DA communications interface of the DCS engineering station with the modern OPC UA communications of the DeltaV PK controller.
Figure 1: The Cogent DataHub mediates communications between the distributed control system's OPC DA server and the PUMA supervisory controller's OPC UA server.
One of the challenges of using a system external to the plant DCS to execute high level automation is safeguarding against communication errors. Writing the wrong value or even the correct value at the wrong time to certain DCS tags can have severe consequences, such as a loss of production or even a plant shutdown. The situations that are most sensitive are when components in the communication chain are started, stopped or modified. For instance, when OPC bridging software is first turned on, a stale value might be rewritten. Or, if the DeltaV PK controller code is modified, outputs may reinitialize to 0 and be written to setpoints within the DCS, leading to a plant upset.
In other applications, DCS level watchdogs and integration logic have to be added on a tag-by-tag basis to ensure proper safeguards are observed. However, with modular automation, there are often many tags to write to, so building safeguards for all of these tags into the DCS is prohibitive. The conditional bridging design outlined in this article embeds the necessary safeguards into the OPC bridge layer. The philosophy is that certain protected tags exist in the DCS, and any write to these tags should be deliberately made by the PK controller logic and not through any accidental circumstance.
Conditional bridging logic
In order to write to protected DCS tags, three conditions must be met. First, the PK controller modular logic is ready to write to the tag group containing the tag; second, communication from the bridge to the DCS and the PK controller is verified to be in a working state; and third, the DCS is ready to receive writes. If not all conditions are met, the bridges to these protected tags will be disabled, and no writes to these tags will take place. However, other non-protected tags will continue to be bridged as usual.
As indicated in Figure 2, DCS_TAG1, DCS_TAG2, and DCS_TAG3 are three protected tags in the DCS. PK_SWITCH_A is a Boolean flag within the PK controller that specifies the modular logic is ready to write to group A tags (in this case DCS_TAG1 and DCS_TAG2). PK_SWITCH_B is another flag for group B tags (DCS_TAG3). A configuration file exists to allow tags to be assigned to groups. This provides protection, so the PK controller can restrict the possibility of any inadvertent writes to tags that aren't required by the active procedural logic. It also simplifies the programming of the modular automation in the PK controller, so the designer doesn't have to worry about aligning out-of-scope tags with the values in the DCS.
Figure 2: Conditional bridging logic (shown here in electrical terms) ensures that no inadvertent tag reads or writes are introduced. (Reference article text for tag explanations.)
DCS_WRITESWITCH is a Boolean tag in the DCS. It can be set by DCS logic and/or by operators. If the flag is false, all writes to protected tags will be prevented. This provides a simple breaker mechanism to disable writes from the modular automation system. Even if normally left in the active position, this is an important safeguard to have available when doing modifications on the system for example.
PK_ALIVE and DCS_ALIVE are heartbeat tags generated in the PK controller and the DCS, respectively. These tags are constantly changing in value (for example, counting up to 60 and then resetting). They're used so the DCS can verify communication to the PK controller and vice versa. Also, the OPC bridge uses them to verify communication to both the PK controller and the DCS. READBK_PK_ALIVE is used so the PK controller can verify its ability to write as well as read from the DCS.
Comm Failure Disconnect is a breaker in the OPC bridge that fails opens, and doesn't close unless both PK_ALIVE and DCS_ALIVE heartbeat tags were recently changed. If either of these tags flatlines for longer than a configurable duration (e.g. 30 seconds), protected writes are disabled. This is an important safeguard to ensure that real (not stale) values of PK_SWITCH_A and PK_SWITCH_B and DCS_WRITESWITCH are being read. It will also ensure that when the OPC bridge is restarted, no unintentional writes to protected tags will occur.
READBACK_TAG1, READBACK_TAG2, and READBACK_TAG3 are used, so the PK controller knows the actual values of the protected tags in the DCS whenever communications are established, even at times where writes to these tags are prevented. This allows the modular automation logic to align tag values with the DCS (for example, set PK_TAG1 = READBACK_TAG1), and decide whether the tags have reasonable initial values before writing any new values to them.
A final important consideration is that protected tags not receive any cached writes. This means that when the breakers are first closed, the value of PK_TAG1 will not be written to DCS_TAG1. Only future writes that occur after the breakers are closed will cause the new value of PK_TAG1 to be written to DCS_TAG1. This will safeguard against any inadvertent writes occurring, such as when the DCS_WRITESWITCH is first enabled.
To realize the conditional bridging functionality, Cogent DataHub uses an integrated scripting language. The features of the bridge were built into a script that reads from a configuration (.csv) file the specifications of all bridges, whether they're protected, and of which PK group they're a member. This script is designed to be reused on all similar projects in the future.
MPA design sequence
This conditional bridge architecture effectively safeguards against all known situations that lead to inadvertent writes. However, it necessitates using a certain design sequence in the modular automation procedures. A typical sequence is:
- Disable all writing from the PK controller side. (Start with all PK SWITCHes in the OFF position.)
- Wait until DCS is ready for writes and any other logic permissives are met. (RDBK_PK_ALIVE is changing and RDBK_DCS_WRITESWITCH is true.)
- Align PK tags with DCS tags. (Write PK_TAGx = READBACK_TAGx.)
- Enable writes from the PK controller side. (Set PK_SWITCH_[active group] = TRUE.)
- Run main sequence logic. (Set PK_TAGx = [New Value] as required, then Monitor RDBK_PK_ALIVE is changing and RDBK_DCS_WRITESWITCH is true, and abort sequence if communication is lost.)
- After the sequence is complete, disable writes from the PK controller side. (Set PK_SWITCH_[active group] = FALSE.)
The main thing to note is the order of aligning DCS tags, enabling the bridge, and writing new tags. Following this order will ensure that the first new value written to the PK_TAGx gets written to DCS_TAGx.
Successful proof of concept
The ability to deploy robust MPA has been successfully demonstrated at the Ingleside ASU, but it’s only the beginning of Air Liquide’s larger vision. We’ve demonstrated that if operators can start and stop equipment in more convenient fashion, they’ll take more steps to operate more optimally when the opportunity presents itself.
For now, MPA operates in an “operator advisory” mode. For example, alerting an operator when an argon column’s product tank is full and the column can be shut down, saving significant amounts of energy. Ultimately, as operations gains greater comfort, the PUMA controller will be able to shut down the column automatically—without operator intervention. And while the MPA’s actions are limited to start and stop for now, the team is eyeing other data-driven tasks that, as we gain confidence in the solution, can be initiated automatically in the future.
