fieldbus_tnail
fieldbus_tnail
fieldbus_tnail
fieldbus_tnail
fieldbus_tnail

Wean process off the redundancy pacifier

Sept. 8, 2006
With Foundation fieldbus, the old redundancy paradigm no longer applies, so where should we apply it to achieve the fault tolerance we need? Contributing Editor John Rezabek provides analysis and commentary.
By John Rezabek, Contributing Editor

While designing our first Foundation fieldbus (FF) segments back in 1999, we had a one-day training session for our engineers and designers. Near the end of the afternoon, someone cracked open the case of the proposed FF power conditioner, which was being passed around for us to heft. We were aghast to find inside multiple integrated circuits (ICs) and (gasp!) fuses. Egad, a single point of failure for our multi-loop segments! There were red faces and veins bulging, and during the ensuing weeks, I was perhaps a bit more unpleasant toward our system integrator than normal.

Our attitude was compounded, to an extent, by the fact that our chosen supplier had relatively modest redundancy inherent in its system. Bulk DC power, controllers, and controller power supplies were redundant, but nearly everything else—conventional I/O and H1 interface cards included—was simplex. We were less confident this supplier, who was new to us, really appreciated the demands our “customer” (the plant) placed on us: basically to “never shut down.”

One way we found comfort was fieldbus backup link active scheduler (BLAS). In theory, if the system had a bad day, control on the segment would continue uninterrupted. However, for this to function, one needs reliable segment power. The “theoretical” segment power conditioners, made up of basic inductors, capacitors, resistors, etc., could be considered a “simple” device, akin to the 250 ohm dropping resistor in a legacy system. But to make them more compact and efficient, manufacturers used ICs. These were not simple devices.

After much agonizing, our supplier saved the day with a redundant solution that was effectively a really simple device.

Knock on Wood
We only put the bulky redundant conditioners on about 20% of all segments—those we considered critical. We used the non-redundant ones, those with the ICs and fuses, on the rest, about 50 segments with between three and 15 devices. Our valve classification guidelines found most of the valves in the plant were “level 3,” which means they could go to their fail positions without causing a shutdown. We applied this engineering judgment because then, as now, redundancy wasn’t free. It not only cost more, it took up more space, supported fewer instruments per segment, generated more heat, and added more complexity.

The irony is—after six years under continuous power and 90% of it running as a continuous process—not one of the non-redundant power conditioners has ever failed in a way that caused a valve to go to its fail position. Nearly half of them did fail, but not in a way that caused more than nuisance alarms or controller-mode shedding. The lesson is: not every failure causes a catastrophe. Many, maybe even most, failure modes don’t result in a process upset. Simply put, all components, especially with improved diagnostics, can have sufficient “fault tolerance” without being redundant.

Redundancy ≠ Single-Loop Integrity
Today, we have a good selection of appliances like redundant power conditioners, redundant H1 cards, and even solutions that accommodate redundant H1 trunks, but they aren’t free either.

Redundancy became commonplace in the late 1980s, when second-generation DCSs, in response to demands for improved fault tolerance from the large process industries, began to offer redundancy at the power supply, controller, I/O, network, and HMI levels. As users, we could specify redundancy and assure our operators that there was no “single point of failure” that would shut down the plant. We justified redundancy’s increased cost, complexity, and system “footprint” in light of the dire consequences of a process shutdown. By achieving fault tolerance for the DCS, we could deliver a solution that was equal, if not more fault tolerant than pre-DCS, single-loop solutions.

I don’t know how many of us in North America’s large process industries still remember, specified, or installed single-loop controllers. Sometimes it seems we have a whole generation of systems specialists, who only remember TDC-3000 was vastly more fault-tolerant than TDC-2000, largely owing to available redundancy at all levels. I was among those who dismissed any PLC or DCS that didn’t offer redundant controllers, I/O, power, and networks for any application more demanding than wastewater treatment or filter cleaning.

I believe that today, with Foundation fieldbus, the old redundancy paradigm no longer applies. If redundancy is free, why not apply it everywhere? Chances are, though, it isn’t free. So where should we apply it to achieve the fault tolerance we need?

Borrowing an SIS Analysis Tool?
Have you noticed the “spurious trip rate” statistic that falls out of SIL analyses? Think about it. Even the most obsessively redundant, bulletproof automation in our discipline, can potentially shut down the plant! Okay, maybe it’s once every 30 or 18,000 years, but it isn’t forever!

Why not use something similar for our “basic” controls? Hey, suppliers! We users need some tools, which have inserted statistics for MTTF and so on, so we can judiciously apply redundancy to components and services where we need it. On my next project, if I mess with all the old “level 1, 2, 3” stuff, I want to be able to tell my project manager, “I know precisely where to apply redundancy to achieve the fault tolerance demanded by operations.”

  About the Author
John Rezabekis process control specialist for ISP Corp. in Lima, Ohio. He has more than two decades experience in process automation in refineries and chemical plants. E-mail him at [email protected].