Open systems threaten smooth migrations

Imagine this: You and your beloved are on a long flight passing over some barren and desolate ice-bound islands en route to an exotic locale, when the pilot announces, “Ladies and gentlemen, you may experience some interruptions to your in-flight movie while we perform an upgrade of this Boeing 757’s control system. This is merely a ‘point’ upgrade, and it’s been thoroughly tested by the good people at Boeing.” I imagine my pulse racing as I gaze down at the hostile and foreboding sea, thinking, “Couldn’t we do this on the ground, for heaven’s sake?” Having personally shut down a chemical plant while performing a similar task, I’ve learned that the benefits of any upgrade need to be substantial to justify the risk of doing so “in-flight.”

Nevertheless, if you’re a modern refinery or petrochemical complex aiming to stretch the turnaround cycle to four or five years, there are bug fixes and enhancements you’d like to apply before, say, 2019. In light of this, our control systems vendors do, indeed, test their install packages for a bumpless transition to the new and improved operating environment. For my system, this can mean deploying the upgrade on hot backup controllers and I/O modules, and then forcing a redundancy switchover, so the active module can get the upgrade flashed as well.

With legacy systems, one’s configuration is largely locked down with fill-in-the-blanks parameters for standardized controls. This makes the chances of a bumpless upgrade much more certain. But because more modern systems have embraced considerable flexibility and customization, it becomes challenging for a supplier to vet every upgrade scenario. In our case, an innocent line of code, putting a controller in manual and its output to zero, had some issues with initial conditions. The result was an unplanned shutdown. That was 15 years ago, and the flexibility and configurability of systems has only gotten better—I mean, worse.

My neighbors imposed a solution (DeltaV) on a systems engineer who had configured numerous similar plants on what was then an Allen-Bradley PLC. His package consisted of numerous custom blocks (blocks that execute some custom code along the lines of Basic or VB) that were constructed to imitate the A-B functionality of that day. Even though both systems could claim adherence to IEC-61131, there was no way to directly translate an A-B program to DeltaV. There still isn’t. And if the systems engineer forgets to validate a variable, check its status, or propagate its status, the behavior under switchovers and upgrade conditions becomes nearly impossible to certify. Indeed, when we do these upgrades “in the hangar,” if you will, we often find out that an online upgrade would have been exciting, if not disastrous.

Even systems and devices that are certified by a fieldbus checkmark cannot have configurations migrated from one platform to the other without nearly total reengineering. Fieldbus function blocks have standard parameters and connections, and are tested in each instance to conform to the FieldComm Group’s specifications. But if I lifted that network carefully off a Honeywell DCS and landed it on a Siemens PCS7, there’s little likelihood it would function without completely redefining and downloading the scheme in the new controller, using its engineering tools—a process that has great potential to be less than bumpless.

If your desire is for standards-based solutions with interchangeable parts, like the bold venture undertaken by ExxonMobil with Lockheed Martin, online upgrades and migrations are but one of the many vexing little “snakes on the plane” that have to be sought out, run into a corner, and bagged. Fear and uncertainty about how solutions from different vendors and/or different generations will behave and interact may send some scurrying back to closed and proprietary. In our familiar world of stovepipe solutions, we can at least dream that there’s someone at our vendor who will be accountable for their deliverables.

Lockheed and its partners are aiming to consolidate the learning from present-day sorties into standards-based solutions, testing and certification. Our present-day standards consortiums have worked very hard to banish the vexing snakes of interoperability, not always with 100% success. But another unexplored (at least in our world) level of achievement will be necessary if we are to standardize on the next level of controls.