Using Modular Procedural Automation to Improve Operations

MPA Goals at FRI

As implemented at FRI, manpower reduction is not one of the goals of the MPA methodology.  In too many global control rooms, manpower reductions have already occurred, and many senior technicians have retired over the last ten years. We believe that experienced operators are a valuable asset, and thus we use MPA to assist, not eliminate, our operators. This assistance is provided by using MPA to share best practices among our operators, allowing the present generation of technicians to learn from our best operators even after they have retired. 

We find that our operators appreciate the assistance provided by our MPA software. It helps them to corroborate their judgments and decisions regarding next steps. Our operators say that using MPA is "like having our best operator alongside them in the control room all of the time."

We feel that the MPA methodology might prevent future accidents at FRI and elsewhere. To verify this contention, we examined how MPA might have been used to prevent past accidents. 

Could MPA Have Averted These Accidents?

The U.S. Chemical Safety Board was formed in 1998 to study major accidents, write reports and produce instructional videos. For several years, FRI has used those videos as focal points for our safety meetings.

At these meetings, one or two CSB videos are typically shown, and the attending engineers and technicians then determine relevance to the FRI operations. After discussion, a list of FRI-related action items is created, with documentation and follow-up to assure that lessons learned are correctly applied.

There were three CSB videos where we concluded that the MPA methodology might have prevented accidents: T2 Laboratories, BP Texas City and Sterigenics.

T2 Laboratories

T2 Laboratories was located in Jacksonville, Fla. T2 produced MCMT, a gasoline additive.  The primary production step was an exothermic reaction in a 2500-gal. reactor that included a water jacket. In 2007, the water supply failed, and a runaway reaction resulted. At some point, the pressure build-up caused the relief valve to blow, but the relief system was too undersized and the reactor exploded.

Details regarding the accident can be found in a CSB video (Reference 1) or the associated report (Reference 2). The video describes a lone board operator who did not know what to do when the water jacket water supply failed. He called off-site engineers for help, and they rushed to the plant, but reached it too late.

If the T2 control room had used an MPA solution, the following would have been possible:

1. Prompts could have been provided to the board operator regarding the back-up water supply.
2. Corrective action steps for this type of a condition could have been programmed as screen prompts or as automated flow rate and valve changes.
3. The software could have set off evacuation alarms.

The knowledge of the plant's most-experienced people could have been provided to the board operator via MPA, particularly as temperature excursions had occurred with the reactor before.  The previous excursions could have been studied using an historian, and the lessons learned could have been implemented within MPA and made available to the operator.

BP Texas City Isomerization Unit

Details regarding this accident can be found in a CSB video (Reference 3) or the associated report (Reference 4). This accident was similar to the T2 Laboratories incident in that a very large burden seemed to fall upon a single operator. Regarding BP Texas City, the report stated, "Restructuring following the merger resulted in a significant loss of people, expertise and experience."  In retrospect, some of that experience could have been captured using MPA and then made available to the operator.

The BP accident occurred in 2005 in an isomerization unit.  When the lone operator came on shift, he had just a single line of input in the log book from the previous operator: "Starting up the Isom Unit."  Unfortunately, the supervisor of the newly arrived operator had been called out of the control room for an extended period to attend to a family emergency. 

The newly arrived operator needed to continue to start the isomerization unit and control two other units, all with virtually no input and assistance. Due to a malfunctioning liquid level alarm, he had no idea whether he was starting with six feet or 175 feet of liquid in the bottom of the raffinate splitter tower. The truth proved to be closer to the latter, and the tower overflowed into the blow-down drum, and the hot gasoline then overflowed and ignited.

If the BP control room had used MPA, the following would have been possible:

1. The newly arrived operator would have had on-screen information regarding all of the steps that the previous shifts had taken, including the exact times of those steps.
2. A safety warning could have been provided on-screen regarding the over-flowing of the raffinate splitter tower (which had already happened a few times in the past).
3. Out-of-date procedures would have been necessarily updated for the sake of the programming of the new software.
4. The MPA software could have shut the unit down in time to avoid the explosion.

If MPA had been correctly implemented, the lone operator of the three units would not have been alone.

Sterigenics Oxidizer Accident

This accident was described in a CSB video (Reference 5) and in the associated report (Reference 6). This accident did not occur because an operator was alone, but because a standardized procedure was not obeyed, and because the operators did not understand the ramifications of diverting from that procedure.

The Sterigenics plant was located in Ontario, Calif. Medical equipment was sterilized in large chambers using ethylene oxide gas. Subsequent to the sterilization, according to procedure, the ethylene oxide gas was purged from the chambers using a three-step process. On the day of the accident, one of those steps was purposely skipped. As a result, a concentrated ethylene oxide gas stream was sent to the oxidizer, and the chamber exploded.

If the Sterigenics control room had used MPA, the software could have provided a message to the operators along these lines, "If Step 2 is skipped, a concentrated ethylene oxide gas stream will flow to the oxidizer and an explosion might result." Upon seeing this message, the operators may very well have taken corrective action.

With MPA, Operators Are Never Alone

Regardless of the reasons, the control rooms of the western world do not seem to be as well-manned as those of 20 or 40 years ago. To address this situation, MPA can be used to transfer the knowledge and experience of the best remaining operators, engineers and technicians to new operators. This makes new operators more effective and efficient. When MPA is correctly implemented, no operator needs to start, run or stop an operating unit alone.

References

1. Explosion at T2 Laboratories, Disc 2, #4, U.S. Chemical Safety and Hazard Investigation Board, Safety videos 2005-2010.
2. U.S. Chemical Safety and Hazard Investigation Board, Investigation Report (http://www.csb.gov/assets/document/T2_final_copy_9_17_09.pdf)
3. Anatomy of a Disaster, Disc 1, #5, U.S. Chemical Safety and Hazard Investigation Board, Safety videos 2005-2010.
4. U.S. Chemical Safety and Hazard Investigation Board, Investigation Report (http://www.csb.gov/assets/document/CSBFinalReportBP.pdf)
5. Ethylene Oxide Explosion at Sterigenics, Disc 1, #12, U.S. Chemical Safety and Hazard Investigation Board, Safety videos 2005-2010.
6. U.S. Chemical Safety and Hazard Investigation Board, Investigation Report  (http://www.csb.gov/assets/document/Sterigenics_Report.pdf)