Systems Integration / Asset Management / HMI / Optimization / SCADA

Process Automation Operator Performance Gets Better

Prioritized Data, Simpler Displays, Human-Factors-Designed Equipment, Fatigue-Reduction Efforts, Alarm Planning and Other Tools Can All Improve Situational Awareness and Operator Performance. So How Much Do You Need of Each?

By Jim Montague

Just as there are many ways to improve process controls and automation, there are many useful ingredients that go into improving operator performance. And, just like mixing up a nutritious fruit smoothie or protein shake, the best recipe for optimal operator performance means finding the tools and methods you need most and will suit you best.

These components include better prioritized data and alerts, simplified displays and software, improved ergonomics, targeted training and simulations, new policies on alertness and physical fitness, and numerous software tools and proactive procedures. All can help enhance operator performance and improve the safety and productivity of process applications, and lately they're being followed by a bunch of other helpful tools and software.

For instance, Dave Strobhar, chief human factors engineer at the Center for Operator Performance, reports it's testing the semantic procedure analyzer (SPA), which is learning software that recognizes and flags terms, updates all applicable documents and makes it easier for operators to organize and update routine and emergency procedures. SPA was developed by Penn State University with contributions from center members Chevron and Yokogawa Corp. of America. The center also just developed its display metrics toolkit (DMT) that lets operators measure their performance with different graphics, which can help them create better tools for individual applications. "We now have a range of tools and methods for evaluating performance, so operators and managers can more easily develop the best solution for them," says Strobhar.

See Also: Immersive Simulator Boosts Operator Preparedness

Collaboration Aids Simplification

One of the best ways to improve operator performance is to get the operators involved early in designing solutions that can meet the specific needs of their applications.

"When we implement interfaces and control systems, we get the operations guys on the team developing the graphics as soon as possible," says Keith Jones, president of systems integrator Prism Systems Inc. in Mobile, Ala. "So after we program the PLCs and DCSs and bring their data to the HMIs, about 80% of the input on what those displays should include and how they should look has already come from the operators. Once operators get some real input on changes they become very proud of them."

Despite these potential benefits, Jones adds there can still be a lot of reluctance to change by older, entrenched operators when Prism is asked to upgrade brownfield applications. "We've seen some really horrible HMIs that pack as much information and color onto one screen as possible. This was done because HMIs and screens used to be a lot more costly, but they make it very hard to pick out what's important," explains Jones. "This is why we began following and preaching the Abnormal Situation Management Consortium's principles on prioritizing data and using minimalist-style graphics and quad-display systems. But even though monitors are cheaper and more effective now, it's still hard to sell operators on them, and management doesn't want to increase their burden or add more training."

See Also: ASM Consortium Process Safety Incident Log

Jones reports that some HMI software can allow users to keep the look of their old, custom screens, but this can make them harder and more expensive to upgrade in the future, so it's better to make the jump to common, standard HMI function blocks and configuration tools, such as Siemens Industry's Simatic PCS 7 and HMI Plus software. "If we can convince a user to change to standard HMI, then we can begin to clean up the graphics and do training on them," says Jones.

Jones adds that PCS 7 has conversion tools for migrating tag databases and hardware-defined tasks from old PCs and unsupported software to new systems, which can greatly reduce the costs of HMI and DCS upgrade projects. "We can even leave some old hardware in place and run it in parallel with a new HMI system for awhile," explains Jones. "Then, if mistakes are made, we can make a punch list and check them by comparing the old and new systems."

Enhancing the HMI

If one of the best ways to aid operators is to fix what they're looking at, are there common recommendations for improving those screens and other HMIs? No doubt.

"Poor basic control performance is rampant, including poor tuning and poorly maintained valves," says Bill Hollifield, principal alarm management and HMI consultant at PAS Inc. and co-author with Hector Perez of the High-Performance HMI Handbook. "This is like sending a race car driver out on the track with lousy brakes. Neither the operator nor the process can perform well with poor basic control. Fortunately, many alarm systems are being fixed, and alarm management has become well-accepted. However, alarms are a small part of the overall operator HMI, and so high-performance HMIs based on new principles for creating screens are needed to correct poor, widespread and ingrained HMI practices. A proper HMI increases operators' situation awareness and their ability to detect and successfully resolve abnormal situations."

See Also: The Eye for Plant Operators' Eyes

Hollifield adds that PAS recently worked with the Electric Power Research Institute to proof-test these high-performance HMI concepts at a large coal-fired power plant using several operators and a full simulator. "The tested HMI included proper hierarchy and embedded knowledge, and we found that operator results were better when handling significant abnormal situations," adds Hollifield. "Besides showing useful information instead of just raw data, displays should be designed in this hierarchy that provides progressive exposure of detail. Displays designed from a stack of schematic designs will not have this. They'll be 'flat' like a computer hard disk with one folder for all the files. This doesn't provide optimum situation awareness and control. A four-level hierarchy, including operation overview, unit control, unit detail and support and diagnostics displays, is desired. For example, Figure 2 is an operation overview from a large power plant. We often hear, 'But it doesn't look like a power plant!' Correct! Does your automobile instrument panel look like a diagram of your engine? The display is designed so important abnormal conditions and alarms stand out clearly." [A whitepaper by PAS on these high-performance HMI principles and the EPRI project is located at] 

Holistic Human Factors

Besides improving HMIs and displays, enhancing performance also means assisting the physical eyeballs viewing those screens—especially when their owners' attention falters and they get tired. Limiting work hours and potentially dangerous fatigue has long been mandated for airline pilots, first responders, truck drivers and other professions, but it's not as well-known or established in many process operations—where many managers and operators rely on too many overtime hours and consecutive workdays.

See Also: User interfaces should empower operators

To remedy these dangerous situations, the U.S. Dept. of Transportation's Pipeline and Hazardous Materials Safety Administration's latest rules took effect in September 2012, and they require pipeline operators to use display graphics that employ human-factor designs; reduce operator fatigue by limiting consecutive hours and days worked to a more normal weekly schedule; establish thorough alarm procedures; document and learn from incidents; and pay fines up to $1 million or more per violation, according to Ian Nimmo, president of User Centered Design Services Inc..

"More process facilities are investing in control room management (CRM) and human-factor principles, and they're taking countermeasures against fatigue, such as deploying 24/7-style chairs, brighter lighting and better temperature control, providing exercise equipment and enforcing regular breaks," explains Nimmo. "They're also monitoring hours worked by operators and calculating the relative risks of incidents, which is going to force some to hire more staff. Besides providing more human-factors graphics, process managers also need to conduct a risk assessment to make sure their operators can handle the number of alarms they're going to get. Then, they need to take a holistic CRM approach, evaluate the roles and responsibilities of their operators and define exactly what they're supposed to do in specific situations, including normal and abnormal operations and in emergencies."

Curing Computer Clutter

While software and computers of all types are indispensable to manufacturing in general and process control and automation in particular, the downside is an increasingly impenetrable jungle of software applications and packages that don't share data fast enough or allow operators to access information they need to make crucial decisions. Fortunately, some solutions are coordinating different types of software and bringing them into more manageable formats.

For instance, Waterford Township's Dept. of Public Works (DPW,) uses many types of process control, maintenance, documentation, asset management and enterprise software to operate its water collection, distribution and wastewater systems, which cover 36 square miles in Michigan and serve about 75,000 residents. The water system pumps and treats about 8 million gallons per day of groundwater from 20 wells at 12 plants and 100 remote sites, and the wastewater system maintains 300 miles of sanitary sewers and sends the township's wastewater 30 miles away to Detroit for treatment.

See Also: Upgrading a Wastewater SCADA System

Bill Fritz, PE, Waterford's public works director, reports that the water/wastewater systems operate hundreds of pumps, valves, actuators, water quality and other equipment, which are monitored by more than 1200 I/O points and networked via Modbus TCP/IP, a high-speed Ethernet backbone with twin servers, and remote radios and 100 live-video security cameras at its pumping stations. All of this data comes into an unmanned cubicle, where it's been managed by GE Intelligent Platform's (GEIP) Proficy iFix software, and then reports, e-mails and alerts are pushed out to DPW personnel on their laptops, tablet PCs and smart phones.

Also, about two years ago, Waterford DPW added Proficy Workflow software, which automates more processes, serves as an electronic user manual, and performs exception-based reporting, which notifies users of problems so they don't have to go find them. For example, the department built algorithms and templates to indicate if pairs of pumps are alternating correctly at 62 of its pumping stations. This allows Proficy Workflow to provide alerts when the pumps aren't synched properly, so individual visual checks by the operators aren't required anymore. 

"It's been our goal for years to get as many applications as possible onto one platform, so we don't have to call up so many different types of software," says Fritz. "But we wanted more than snapshots of what's going on, so we're installing new Proficy Vision software, which brings all our separate software programs into one customizable dashboard, and allows us to bring up and quickly scroll through all our individual programs that we couldn't see all at once before. Now our office managers and 56 field staffers can view water operations on the SCADA system, HVAC data, vehicle locations and status, security camera feeds, work orders and other information at the same time (Figure 1). So if there's a water main break, they can trace it with GIS, isolate it faster, check affected pump rates and tank levels, find the closest person and vehicle, tell operators which valves to close, and get specification data into the right hands more quickly."

See Also: Wireless: A Field Guide to Industrial Wireless

Don Busiek, GEIP's general manager for manufacturing software, adds that Proficy Vision was just released in January, and while it renders several types software on one screen, its counterpart Proficy Mobile software will allow users to access SCADA data on tablet PCs and smart phones. "This all about understanding how operators navigate and what they need, and then coming back and providing the right data to them at the right time," says Busiek.

Training, Software and Other Tools

Apart from all the displays and technical solutions, another of the best ways to empower operators is still good, old, thorough, in-class or on-the-job training, which is often aided by online curriculum and simulations.

For instance, to prepare for commissioning and start-up on its Angel platform on western Australia's Northwest Shelf, Woodside Energy Ltd. recently decided to use UniSim operator training simulator from Honeywell Process Solutions to instruct operators how to run its Experion C300 Process Knowledge System (PKS) and quickly diffuse potentially unsafe situations. In fact, Woodside reports employing UniSim before commissioning prevented at least five process trips.

"The Angel training simulator was identical to our real Angel control room panel and enabled our operators to practice on the new C300 system," says Dustin Taylor, Woodside's process engineer. "We also used UniSim Design to model the process, which provided dynamic process behavior for training our panel operators. UniSim increased the competence and confidence of our staff when dealing with unsafe scenarios, plus we reduced the number of trips."

Andrew Ogden-Swift, director of technology strategy at Honeywell Process Solutions, adds, "There are several approaches to training operators including classroom, on-the-job and using standard and custom simulators. The key to proper simulation is defining the competencies needed for operators to perform their roles effectively, while designing training solutions to achieve required competencies."

Similarly, Glenn Goldney, global programs manager for Rockwell Automation's Global Workforce Solutions (GWS) division, adds that, "Operators have been a very neglected population. Not only is there a widespread skills shortage, but their job descriptions are hard to define because of their multiple crafts and roles, and even those existing job descriptions are changing rapidly. This is why training is more important than ever for developing and retaining talent. This is why GWS also helps users segment their job structures and redistribute some responsibilities, so everyone can be more effective. For instance, we recently helped Alyeska Pipeline Service Co. develop a custom job interaction and e-learning solution for operators at its emergency, back-up power plant in Valdez, Alaska, which serves transfer facilities at the end of the Trans-Alaska Pipeline."

Likewise, AES Wind Generation, a subsidiary of AES Corp. in North Palm Springs, Calif., recently installed 49 of Mitsubishi Heavy Industry's (MHI) 1-megawatt wind turbines at its Mountain View IV wind farm and implemented Iconics' Genesis64 SCADA system and Hyper Historian, which are networked via open protocols such as OPC, take in data from Modbus TCP/IP and MHI CNET interfaces and provide AES' operators with immediate and aggregated data for monitoring and reporting. Genesis64's global aliasing function also allowed the wind farm's staff to configure controls for one turbine and then quickly import the same I/O and other settings in the other 48 turbines.

"Many operators are increasingly overwhelmed with data because each is typically being asked to manage more systems," says Russ Agrusa, president and CEO of Iconics Inc.. "Operators must be able to integrate disparate information and software into standard, common interfaces. They need tools that can dig through "big data" and allow priority alerts to bubble up to the top."

Seeing in the Future

Naturally, as most human endeavors migrate to tablet PCs and smart phones, some even cooler tools are beginning to pop up to aid operators. One of these futuristic, collaborative tools is Mobile Voice and Video from Emerson Process Management. This wearable, video-conferencing system allows field operators to point a hands-free camera at problematic equipment, and confer with technical experts at a distance (Figure 3). It was developed with components from AudiSoft and Frontline Communications.

"Mobile Voice and Video has full, high-definition video, but it also makes voice communications a priority," says Neil Peterson, Emerson's senior wireless marketing manager. "This means it can switch to fewer frames-per-second video based on available WiFi or cellular bandwidth to keep conferences up and running."

Better Interfaces = Better Operators

There are a few basic steps needed to upgrade human-machine interfaces (HMIs) and their software, so they can contribute more fully to helping operators and their applications. Keith Jones, president of system integrator Prism Systems, reports these are some of the most crucial:

  • Involve operators up-front, discuss why an upgrade is important, secure buy-in and allow input on planned changes.
  • Decide on what HMI platform to use and seek proficiency in it by aligning with a system integrator and supplier partners.
  • Renew focus on overall business goals, understand required operations and define or redefine displays and graphics functions to best serve those tasks. For example, don't make displays that merely recreate P&IDs, but instead evaluate the manufacturing process to determine what operators truly need to know, ask them what screens they use most and build HMIs based on that data.
  • Draft a project team and then cross-pollinate by sending at least one person from the end-user's side to the system integrator's side and vice versa.
  • Hold regular visits and face-to-face and WebEx meetings to update participants on the project's progress, and seek new input.
  • Develop HMI project scope, plan and acceptance schedule, including layout, graphics placement, navigation procedures, alerts and alarms, user identification and other security requirements.