By Rich Merritt, Contributing Editor
Back in the day (circa 1979), the engineers at Electronic Associates (EA) in Monmouth Beach, N.J., let me cause my own “Three-Mile Island” incident using one of the simulators they built to train nuclear plant operators.
Those 1979-era simulators consisted of a complete replica of a nuclear reactor’s control room, with all the single-loop electronic controllers and panel readouts connected to a minicomputer simulating the reactor. The instructors at EA showed me how to “scram” a reactor. They led me through the entire process, saying, “Shut off the feed-water pump. Close this valve. Ignore that alarm.” I uncovered the core, and scrammed the simulated reactor. Great fun!
At about the same time, the movie The China Syndrome was released, and scared audiences with the near-meltdown at a fictitious nuclear plant. The hero, a control engineer named Jack Godell, keeps telling everyone there’s a problem with a pump, but no one listens. If the Godell character had had modern computer modeling and simulation software, he could have fed all the vibration data into a simulator, the computer would have predicted the outcome, and he wouldn’t have been machine-gunned for his troubles.
Fast-forward 30 years, and the folks at video surveillance vendor Longwatch have built the simulator Godell needed. Longwatch president, Steve Rubin, explains, “We took some clips from The China Syndrome, and put them on an HMI screen to illustrate what such a simulation might look like to an operator.”
In the simulation, the HMI screen first shows normal operation, then an alarm goes off, and the video switches to a “live” camera in the pump room. We watch as the pump shakes itself to death and falls to the floor. [Go to www.controlglobal.com/0810_Simulation.html to see this video and others. The direct link to the video is www.controlglobal.com/0810_SimChina.html.]
The pump animation in The China Syndrome was the result of 1970s movie magic, using mainframe computers with about the same computing power as today’s PlayStation 3. But we’ve come a long way since then, with video gamers leading the way. For example, in the new iRacing racing simulator (www.iRacing.com), the builders laser-scanned in each race track to “replicate the personality, eccentricities and challenges of the track with mathematical precision.” [ To see this realistic simulation in action, go to www.controlglobal.com/0810_SimRacing.html.]
So the question is, can we use the same technology for future process modeling and simulation software? Why would we want to?
Getting Down With Graphics
HMI screens have improved in recent years, mostly in the area of graphics. Today, process control equipment and icons are downright realistic. Marcia Gadbois, vice president of business development for InduSoft, says these HMI graphics lend themselves to modeling and simulation. “When running a simulation, we can make the valves open and close, the pumps start and stop, tanks fill and empty, and show variables changing on the screen.”
But going much beyond this level of graphics in the process control industry is probably unlikely. Companies are not going to laser scan their process areas, so they can be animated on an HMI screen. Though CAD packages can certainly show a 3-D representation of a process unit, what would be the point? There really isn’t anything to see. In a process plant, most action takes place inside things like batch reactors, tanks and distillation columns.
N-D Better Than 3-D
Robin Brooks, managing director of process improvement software vendor Curvaceous Software, advises thinking beyond 3-D graphics to multiple dimensions (N-D). “We see the future as N-D with process operation and alarms and alerts tamed and seen in context,” he predicts. “N-D lets operators see many more variables with alarms and all the interactions between them, and is capable of eliminating alarm floods. How much interaction? Probably less than today, because controls will get better and better, and operators will be forced out of the loop. Start-up and shutdown will be computer-controlled for increased safety. Their role will be limited to deciding what to do in cases of genuine equipment failure only.”
In Curvaceous’ N-D simulation of a multi-phase batch process, which runs at about 100x real time (batch processes are slow in real life), black dots are variable process variables, red indicates an alarm, and blue is corrective advice for open or closed-loop use. Green lines are multi-variable control limits. An operator just has to keep the black dots inside the green lines for a perfect batch. [Go to www.controlglobal.com/0810_SimCurv.html to see this simulation.]
Longwatch’s Rubin agrees with Brooks about operator functions. “Watching an HMI screen has been described as long periods of boredom interrupted by moments of panic,” says Rubin. “If an alarm occurs, operators leap into action. If the plant has video connected to the HMI screens, a camera shot can be instantly put on the HMI screen, so the operator can see what’s happening out in the process unit. It might be a leak, a tank overflowing, or some other visual event.” Rubin thinks video is a better solution than animated graphics for simulations. “All these video snapshots can be archived for future use in a computer model or simulation,” he says. “Then, when the simulation says a tank will overflow, a video snapshot of the tank actually overflowing could be put on the screen.”
Tim Gellner, director of operational consulting at systems integrator Maverick Technologies, Columbia, Ill., says better graphics with higher resolution aren’t needed. “The sole function of graphic displays is to provide the operator with timely, easily digestible information about the process being controlled and to facilitate the operator’s ability to make the correct decisions when a process upset occurs,” says Gellner. “Unless better graphics can enhance this functionality, they may just be glitz. On the other hand, when using simulation tools for analyzing work streams and product flows through a facility for optimization purposes, the more realistic the graphics, the better the simulation, and the higher the impact.”
Greg McMillan, principal consultant, Emerson Process Management, adds that, “Graphics aren’t as important for process and control system design studies, but they sure help sell the idea and impress users—particularly the ones holding the purse strings.”
Jonathan Phillips, Arena Simulation product manager at Rockwell Automation agrees. “Simulations with detailed animation aid in selling whatever is presented,” he says. “Also, they can help when the audience may not have a great understanding of the system or isn’t focused on its minute details, such as an engineer presenting a proposal to accountants about a capital expenditure.”
Discrete automation simulations generally have better graphics than process modeling. “Discrete automation simulations benefit from detailed visualization,” says Hosni Adra, product manager/partner at CreateASoft. “Graphics help identify where a robot can be positioned in relation to parts and other equipment to improve efficiency and performance.”
For example, CreateASoft built a Simcad simulation of a 300,000 sq ft warehouse. “The company needed to improve efficiency of its put-away process with available material handling equipment,” explains Adra. (Go to www.controlglobal.com/0810_SimWarehouse.html for this video.)
Proposed changes included moving racks further apart and losing one rack of capacity, but the Simcad simulation showed it would work. So, the company reduced head count and improved velocity, and reduced put-away time from 7 hours to 3.5 hours. In the video, you can see simulated robots, AGVs and people scurrying around the warehouse.
Likewise, Molded Fiber Glass (MFG) in Ashtabula, Ohio, produces parts from polyester resins and fiberglass reinforcements (FRP). Robert Morrison, MFG’s founder, says they used Simcad to model a composites manufacturing plant. “The model required tracking multiple variables including cycle times, labor costs, and manpower utilization,” he says. “The goal was to detect and identify bottlenecks, and reduce overall production costs. Using Simcad’s graphical interface, MFG built the model representation of the manufacturing line, and then added manpower allocation and cost detail.”
Using results from Simcad’s animation and line analysis, MFG found its bottlenecks, adopted a proces flow that saved one man hour per part, maintained current production rates, and reduced labor required at the plant.
Driving a Simulator
Tomorrow’s engineers, accustomed to the fast response of computer games, will be bored with traditional process simulators.
“Traditional simulation requires an operator to generate and pass code to a computing engine, which then has to be compiled,” explains Adra. Once the simulation starts, changes can’t be made in a traditional simulator.
The first step in a dynamic simulation, then, seems to be automating the way a simulation is set up.
“ExperTune’s PlantTriage software uses active-model capture technology to develop process models from normal operating data,” says George Buckbee of ExperTune. “The software monitors the process 24x7, finding, qualifying and storing models in a database, so they’re ready for use. This software helps engineers develop a detailed understanding of the dynamic process response. With these models, they can choose control strategies and tuning for optimum process results.”
For instance, Eli Lilly pulled historical data, and fed it into ExperTune’s PID Loop Optimizer to develop dynamic process models of a bioreactor. These model pairings were then used to develop an MPC controller. It was tested offline in a simulator before implementation in the real reactor.
As for “driving” a simulator, many “traditional” simulators can do it. Gellner says, “Even in simple loop-back simulations you can modify parameters on the fly by testing controllers’ responses under well-defined and documented scenarios.”
However, vendors don’t promote dynamic simulation very well. Perhaps because it’s new ground, and many vendors are busy preparing next-generation simulators. Vikas Dhole, vice president of engineering product management at AspenTech, agrees this is possible, but yet scarce. “On-demand decision support provided via real-time simulations is available,” Dhole says. “The challenge is combining a simulation tool’s technical capabilities with a graphical interface that can interpret data quickly and easily. Though these two elements are converging, less than a handful of commercial players are able to support such product development, and even fewer have something available currently.”
PepsiAmericas used simulation animation to check the operation of a new pallet line at its plant in Austin, Indiana, before it was built. Using Rockwell Software Arena simulation software, PepsiAmericas created an initial design concept prior to installation. The company then used the Arena software to calculate the many variables that can occur in the process during production. The animation allowed engineers to view and manipulate the system under a variety of conditions. The team was able to run the model with different scenarios using production variables such as flow rates, product volume and conveyor speeds. With each scenario, the team could evaluate system efficiency based on the output from the Arena software.
“We find the most value for dynamic simulation in training,” says Alex Johnson of Invensys. “We can run what looks like a real plant for the operators, inject faults, speed ups, slow downs, and then do an instant replay to show them what went wrong.” It also works during startup. “If we can convince a client to use Sim-Sci dynamic simulations, we can dramatically shorten startup times,” Johnson says. “It’s a good way to discover that equipment isn’t hooked up correctly, because the process doesn’t respond the way it should. We often find valves that are plumbed or connected incorrectly.”
Honeywell's UniSim Operations can do this now, says Stevenson. “With UniSim Operations, a duplicate of the operator control console used in the plant can be interface to the dynamic simulation,” he explains. “The operator can interact with the dynamic simulation just as if it were the real plant, getting realistic process responses to control actions.”
Many of the more important processes are too slow to run in real time says Greg McMillan, principal consultant, Emerson Process Management. “Distillation columns take hours or days to line out, most chemical batches take hours to complete, and many biochemical reactor batches take days to weeks to complete,” he explains. “A cell culture bioreactor’s cycle time typically ranges from 10 to 20 days. My simulations of bioreactors embedded in a DCS to create a virtual plant can run 1000x real time so these batches take 15 to 30 minutes. The user can change parameters as the simulation progresses, and stop and restore runs in DeltaV SimulatePro to create a true interactive environment. Another powerful feature involves the fast playback—10,000x real time or more depending upon data density and data processing capability—of scenarios produced by models already running faster than real time.” Existing plant data can also be played back at high speeds for data analytics and diagnostics.
“Dynamic system simulation requires more sophisticated models, but yield much more information,” says Javier Gutierrez, product marketing manager, National Instruments. “Another vector to consider is how the simulation is going to be executed: offline, where the system is fed with all the inputs and all the results collected for analysis; and online, where the execution of the simulation is done in real time--similar to videogames. We found that real-time simulation is one of the most appealing for new areas, as it allows training operators and changing design parameters on the fly to quickly experiment with parameters.”
Samsung Techwin, Seoul, Korea, used LabVIEW dynamic simulation, to develop gas turbine engine controllers. “By mathematically modeling the gas turbine engine, we were able to calculate the engine performance parameters,” says Moon Sohk Chae, at Samsung’s Power System Research Center. “We then converted those results into actual physical signals that we input and output to the gas turbine engine controller. Through trial and error, we tested the engine controller hardware and algorithm, improving reliability, reducing debugging and calibration time, and helping us prevent unanticipated controller malfunctions.”
The simulator has a virtual engine simulator, gas turbine engine controller, control algorithm, and a simulation server for the operator interface. “Using the simulator server program, the user can perform such user interface tasks as temporary engine status modification, simulation setting modification, simulation start, pause, and exit, and ultimate engine status setting,” notes Moon Sohk Chae.
“One of the most interesting features of online simulation is the capability to easily replace the simulated system with the real one,” notes Gutierrez. In a simulation of a helicopter control system, engineers used LabVIEW to solve a problem with three interacting PID controllers. “Finding the right system gain is a real time-consuming problem,” says David Whatley, CompactRIO software group manager. What they did was to monitor the responses of a real helicopter, put the data into the simulator, created a model, and ran tests. The simulated control had ten times better response than the helicopter’s standard control system, and the final control algorithms in the simulator could be loaded directly into the helicopter controller. The video (www.controlglobal.com/0810_SimHeli.html) explains how this was done.
“Simulation for training has the greatest need for virtual world modeling,” says Buckbee. “In fact, multi-player role-playing scenarios are now being used to train operating teams for nuclear reactors. The ability to create scenarios and characters on top of a high-fidelity real-time model allows operators to train quickly and to practice for abnormal situations. Such simulations are already in use in many plants and industries. Static and dynamic models are used to predict the effect of changes. Many chemical companies have fairly complete process model simulations.”
Running Simulations against Live Data
Suppose you could do a “what if?” while you were running the plant. “What if we took pump P163 down for emergency maintenance, and routed the piping through valves V234 and V567? What would that do to the process?” If you could run such a simulation, it would tell you what the outcome might be before you made the physical change. This, of course, requires plugging all the current, real-time process conditions into a simulator and running it in “faster than real time” to see.
“The ability to run process simulations against real-time plant data would provide insights into the operation of the control system which are typically not available in other methods of simulation, and would allow the controls and process engineers to make adjustments or fine tune the system,” says Maverick’s Tim Gellner.
Even better than that, what if a simulator was running next to the process control system, and keeping an eye on what might happen an hour, day or week ahead? It could alert you to possible future problems. All this is possible today.
“They can do this now,” says Vikas Dhole of AspenTech. “They can also map multiple simulations in parallel to optimize processes that produce better products, while running a safer plant.”
Stevenson of Honeywell says live data can be used with process simulation to help predict what will happen next or even to help drive the operation to a desired condition. “In fact we actually do this now,” he says. “Current data can be used to synchronize a dynamic model to current operation, enabling the user to vet proposed control moves in order to look ahead at where the process is going. Historical data can be used to synchronize a dynamic model to historical operation enabling the user analysis past operation and test proposed solutions to problems.”
“We are seeing more industrial applications where live-data simulation is being applied, says NI’s Gutierrez. “One example would be Siemens, where they use simulation to test the controllers for a high power turbine to generate electricity.
“Before delivering an automatic steam turbine controller to our customer, we test the turbine regulator hardware and software,” says Eckart Brackenhammer of Siemens. “We test the automatic controller under different operating conditions to detect disturbances in the system. Often, customer representatives bring their own test points into the inspection report. We deliver the turbine controller only after a successful test conclusion.”
Siemens can test the different operation modes, including the turbine starting within a specified revolution rate, the turbine operating under load or pressure control, load decreases under idle operation, load removal and turbine shutdown. In addition to the actual functional test, the turbine simulator can aid in parameter tuning and optimization in the field prior to operation.
“The simulator consists of two separate programs and computer systems,” says Brackenhammer. “The first is the real-time system running a real-time OS that calculates the model. The second is a laptop with Windows 2000 and LabVIEW, which contains the user interface and the model parameter computation. We connect the host and real-time system via standard TCP/IP.”
Greg McMillan says you can make a video game out of such modeling, and foster competition among operators. “You want to be able to run the model in real time synchronized and adapted online with the process,” he says. “Then you can use process performance metrics, data, analytics, and diagnostics online to spur competition between operators on the same equipment but on different shifts or on similar equipment, such as multiple parallel reactors. By synchronizing and adapting a model online to the process, it can then be taken offline and run much fast than real time to predict future results.”
So we are back to video game technology.
The Gaming of Modeling and Simulation
How much of current game technology will we see in modeling and simulation? “AspenTech foresees a major shift coming as process simulation and optimization solutions catch up with the gaming generation of new engineering professionals,” says Vikas Dhole of AspenTech. “They have come to expect the same easy to navigate and intuitive interfaces as they’ve experienced using game consoles like Xbox and PlayStation. We are already starting to see new simulation solutions that can provide model-based decision support, and online demonstrations or viewlets that allow new users to get immediate training on the products.”
Immediate training would certainly help. In our June 2008 survey of trends in the process control industry, 30% of those who have installed loop-tuning software (36% of the total number surveyed) say they have it installed, but don’t use it as much as they could. A quarter of those who use simulation software think they don’t use the products to the extent they could. Maybe it’s the training. Or lack thereof.
We might even see game accessories. “Video will be abundant compared to today’s usage and this may be the area where joysticks, wrap-around visors and the like come into their own,” says Robin Brooks. “These will reduce the need for field operators to go into the plant itself, which probably will be mandated by OSHA, legislation or insurers.”
“Improved operator interfaces can enhance an operator’s ability to diagnose and react to a situation,” says Phillips from Rockwell Automation. “One thing that needs to be considered is the fine line that exists between too much information and not enough information.”
Longwatch’s Rubin agrees. “The benefit of video is that it can transfer information faster, without increasing the complexity to collect and deliver that information,” he says. “Consider the alternative: bunches of sensors and hours of operator training needed to interpret those sensors.”
Alex Johnson of Invensys also sees increased use of video and webcams. “It might be possible to use gloves and goggles to give operators a real feel for the plant,” he says. “Some of this technology is already in use in military and aerospace simulators.” Brooks says we will see a new generation of process historians emerge that contain all the data necessary to allow the operator, or anyone else, to rewind the process to a time in the past and see the process play forward from there exactly as it happened then.
“We have asked ourselves these questions,” notes Stephenson of Honeywell. “Would an operator trainee be better trained if we incorporated tactile, vibration and sound feedback in the UniSim Operations product? The answer to this question may be yes if the control room is not fully isolated from the plant. We do see some value in providing a 3D rendering of the plant, enabling an operator in training to ‘walk’ through the plant. A 3D rendering of a plant may also help a trainee operator to interact with a trainee field operator who is in the plant virtually. In fact, we are following a couple of threads in this regard: a research proposal with the ASM Consortium and a customer that has shown specific interest.”
New multi-core computer technology will help, too. “As simulation models become more complex and detailed, the time it takes to execute a simulation increases exponentially,” says Gutierrez. “The challenge is how to leverage multi-core processors to run applications and simulation in a parallel fashion.”
The answer is simple, in my opinion: Just mimic whatever game technology does with multi-core processors.
Rich Merritt is a Control contributing editor