By Ian Nimmo
Of all the changes in the process control industry, none may be more dramatic than the changing role of the “operator.” His (or her) job has changed numerous times in response to technological advances, but we may be looking at the greatest change since the industrial revolution—the separation of field and control from operations.
Before you write me off as a wild-eyed crazy, let’s step back and see how we’ve gotten to where we are now.
In 1953, only 100 computers existed in the world. IBM developed the 701 EDPM, which the company claimed was the first commercially successful general-purpose computer. Developed as part of the Korean War effort, the 701 was available to early computer users for $15,000 month.
Process manufacturing embraced this new technology, which would provide integrity for early process control computers, via core memory. The older engineers among us will have fond memories of loading an image and hand-booting the early PDP 11 machines. They always survived a power outage, and data corruption was rare. Since then, with every advance in computing power, the role of process control operators and engineers has changed.
When I started my career, a typical plant had multiple instrument and electrical engineers. However, 20 years later a maintenance manager, typically a mechanical engineer with responsibility for all of the instrument, electrical and mechanical operations, replaced all these people. Today, a new player—the control engineer—rules.
Operators’ roles have evolved as well. In the early days, local control shelters accommodated operators, who worked together as a team, sharing all the duties. In some cases, the most experienced operator or chief operator staffed the panel, but this person often had some outside duties too.
The panel operator worked with pneumatic instruments and controllers that provided information through clockwork chart displays. New hires were often “chart changers,” whose job every shift was to go along the panel, replace the chart paper and fill the color pens up with ink. But over the years, electronic single-loop, multiple electronic, PLCs and, eventually, distributed controllers replaced this technology.
The roles and responsibilities of panel operators have fluctuated for many years. Now with the popular trend of specialization and centralization, they are finally standardizing. However, for how long is probably a good question.
As the role of the panel operators has evolved, their home, the control room, has also been evolving from the uncomfortable and decidedly “people-unfriendly” brick shelters in the middle of the plant to the modern “theater style” control rooms properly designed for their occupants.
Figure 1. The old-fashioned control room panel wall has been replaced by banks of monitors.
Meanwhile, the roles and responsibilities of traditional field operators may not have changed as dramatically. They, for the most part, still work in buildings in the middle of the process. However, these too are often now custom-designed, blast-resistant buildings with limited controls.
The field operator handles fewer manual operations, as most of the equipment is now fully or semi-automatic. We still have equipment that requires manual starting—big valves that need turning and safety-related equipment that requires manual inspection during startup.
The Control Room/Field Split
Many of those responsible for organizing their process control systems still have mixed feelings about roles and responsibilities and how the work should be split between the field and control room. They often ask “should all jobs be rotated or should specialization and skill divide these roles.”
For a better perspective, let’s review this evolution and examine what the roles and responsibilities should be.
We also need to put these roles in context. If you have attended one of the popular Operational Excellence seminars (OpX), you will have heard the instructor say that the most important and mission-critical roles in the plant are the process control operator, then the field equipment operator, followed by their shift team supervisor.
The process control operator or console/panel operator is the one person who has the most influence on the success or failure of your production process. This person can make a profit, or be responsible for major loss. He or she can influence product quality and the cost of manufacturing, as they can influence energy usage and the cost of utilities.
A good friend of mine always tells plant managers that these people are so important to you that you should send a limousine to pick them up each morning and take them home at night. You better treat them right while they are at work and provide them with the best environment possible to ensure that they stay awake, alert and motivated for work.
Everyone laughs at this—especially the operators—but behind this joke is an important message, one that challenges using process control operators as janitors and cleaners. Permanently monitoring the process is a best practice. An operator needs to maintain situation awareness, and any distraction, including other duties, makes it impossible to track the process and anticipate abnormal situations.
Through this examination of the evolution of the operator’s roles and responsibilities, we can identify some of the gaps in the new practices which many companies struggle with on a daily basis.
At one time, every operator was an outside operator. His or her role was to monitor and control equipment and coordinate maintenance activities. He took samples of the product and tested them for quality and against a specification. She had cleaning and minor maintenance duties, such as greasing and draining lines.
These duties have not changed too much, except we have automated a large proportion of the manual manipulation. The remainder are too difficult or expensive to automate.
In the past, the workflow was organized around the experience of the team. The new folks did the lab sampling and ran errands while they observed and learned each individual job.
The introduction of instrumentation technology moved control into local control rooms, but the workload was still shared, and operators would work inside and outside duties. As more technology was added and the complexity of controls increased, it became more economical to dedicate a single person to the control room.
This became the job for either a dedicated panel operator or a new role for the chief operator. Many companies liked the flexibility of having the chief working inside and outside, so they created a new job called panel operator. The work allowed some flexibility, and it was common for this person to go outside to manipulate equipment close to the control room.
So now three distinct operator roles exist: the field operator, the panel operator and the chief operator. Each position was seen as a career advancement. It was valuable to the team that as many of the field operators as possible became qualified on the board so they could cover for vacation and sickness. It was also valuable that an operator had a complete understanding of both inside and outside roles.
Situation Awareness
The advantage of having dedicated panel operators was that it allowed them to keep an eye on the big picture, and they developed exceptional “situation awareness.” Operators used good pattern recognition skills to monitor many process trends, developing techniques for tracking the difference between good and bad values. They constantly used real-time and hourly averaged trends. This only is achieved when the operator is aware of the immediate, intermediate and long-term goals, plans and real-time events.
To protect the panel operator’s situation awareness, the chief operator handled disturbances by outside influences, such as maintenance people requiring permits. Over time, this changed, and maintenance people, managers, engineers, supervisors and a host of other people invaded the control room, causing major distractions. This was the start of the loss of situation awareness.
At this time, the instrument panel was designed around tasks, allowing the panel operator to stand at one section of the panel and take control to handle any disturbance. A bigger event required the operator moving back and forth down the panel. As this type of event unfolded, other operators and supervision often stepped up to the panel, and the controls were partitioned based on peoples knowledge and experience. When an operator had to step outside to manipulate or check equipment manually, someone else assumed responsibility for the controls on his section of the panel.
Technology continued to evolve. Electronic controllers replaced the pneumatic ones, but the roles and responsibilities did not change. That happened only when the physical controllers were replaced by a computer and software.
The distributed control system (DCS) replaced the panel and now provided a keyhole view of the process, isolated through a keyboard and computer screen. Several screens were provided, but at some point, it became difficult for operators to multitask the control function.
The early introduction of the DCS still allowed the operator to work inside/outside duties, but now the operators lost their good situation awareness, and designers compensated by introducing more and more alarms. Operators soon lost the ability to foresee the intermediate and long-term and became more reactive. The early DCS software handled trending very poorly and soon trending was rare and alarm floods common.
The role now changed from a panel operator to a console operator. Most companies tried to keep the roles and responsibilities identical to the previous panel operator model.
The more complex the DCS became, the harder it was to monitor that big picture and for operators to maintain their situation awareness. They were out of the loop.
It became apparent that some operators had natural ability and often liked working with the DCS, while others did not. Many companies have recognized that the evolved roles have different skill sets, and that they need to hire for specific positions.
Sounds like a perfectly reasonable approach, doesn’t it? However, the practical aspects of providing overtime cover for sickness, holidays and other events that take a qualified console operator away from the job created a problem with coverage. To get around this problem companies decided to rotate operators and keep them qualified in both inside and outside duties.
This all happened, often in the same control rooms, where the pneumatic controls once dominated. However, due to this loss in situation awareness and and rise of a reactive operating stance, the potential for human error and accidents dramatically increased. Regulators responded by requiring companies to move non-essential personnel outside the process area.
Tradeoff
The console operator became one of those casualties. A safer working environment replaced the benefits of:
- Face-to-face communication,
- The ability to use other human sensors (other operators)to detect an abnormal situation, such as hearing plant telltale sounds or smelling a potential leak,
- Having the supervisor in close proximity to both groups of operators.
At the same time, the new environment did provide some new and exciting benefits, such as:
- The ability to put all the control operators together and organize them based on communication and collaboration requirements,
- The reduction in disturbances, such as maintenance and permitting, typical in the local control rooms,
- The ability of console operators to concentrate on shutting the unit down during an emergency, without the consequences or bad effects of that emergency, such as a toxic release, fire or explosion.
As companies have tried several operating models, pace-setting organizations have exploited the benefits, worked through the negative issues and mitigated many of the problems using unique solutions.
New Guy in Town—The Process Control Operator
This has resulted in the next phase of the evolution, the change from console operator to process control operator. This new role is not just about monitoring the instruments with computer screens, but also keeping track of the productivity of the process.
The job has changed and not only requires a more detailed understanding of the process and equipment, but also of the control strategy through regulatory control, advanced predictive model control and optimization.
This operator is responsible for starting and shutting down the process, controlling it in a steady state to ensure safe production, while managing optimum performance and being conscious of the operating costs. This person has sole responsibility for managing abnormal situations and, when required, making the decision to shut the plant down safely.
Every operator who performs this job must be fully qualified and competent to carry out all the expected duties. He or she must be competent in teamwork and able to provide direction and leadership to the outside field operator during diagnosis and recovery events.
The role is not just monitoring and reacting during an abnormal situation. A good process control operator will maintain situation awareness of the processes under control. He or she normally will be managing multiple processes on multiple units.
Continuing Evolution
As the operator’s job continues to evolve, many experts believe the traditional field operator job is changing also. The traditional activities we have discussed will still be a major part of their jobs, but they will assume more responsibility for the reliability of the equipment.
We are already seeing field operators monitoring and tracking rotating equipment bearings using handheld vibration monitoring computers, studying valve performance, predicting equipment failures and doing more in the coordination and setting of maintenance activities.
The role is changing to that of an equipment specialist and may become part of the maintenance organization rather than the operations team.
The challenge many companies face is how do we get there from here? How do we maintain the field knowledge of the process control operator and reinforce the mental model of the process and equipment to be effective.
Today, companies are struggling with these issues, especially in light of the aging workforce and loss of essential experienced personnel. We are seeing, the introduction of 50% new operators into this environment, along with 50% new maintenance people at a time when we are trying to merge two disciplines. There is also the impact of adding large numbers of young managers, supervisors and engineers as they too have large numbers leaving the workforce.
What we do know is that we cannot address this issue by automating our way out of it. We have already witnessed the impact of increased system complexity, loss of situation awareness, system brittleness and increased workload at inappropriate times.
What of the future? Technology must be exploited in accordance with human limitations. We need a different approach to technology. It needs to support the way people work.
We have a new model for each of the operating roles. We know how important they are; we also know of the challenges we face with training new people, so we must start using the technology more effectively. We have had simulation technology available for many years, but we have not used it because of the cost associated with purchase and maintenance.
Our challenge is to make it cost-effective and to leverage it properly.
How does the traditional shift team supervisor now fit into this organization? We have seen the role splitting with the creation of a dedicated shift team supervisor just for the process control operators. This role has the responsibility to optimize the process using this team and being the front line manager for disturbances.
Supervision in the field will be essential. As new people are introduced, we will need good training here also. The role of this supervisor will be to manage the transition and ensure people are working safely. This supervisor will also have the responsibility of first responder in an emergency and will not only manage his troops in the field but will coordinate emergency services and maintenance as required.
Control rooms will be centralized and away from the dangers associated with processing highly hazardous chemicals. The field will have facilities, but they will be modular, blast- resistant field shelters with limited control functionality and will be focused on supporting equipment reliability.
This is probably the greatest change to operations since the industrial revolution. It will have social and economic repercussions. Some will be reluctant to see the separation of the field and controls from operations, but we should examine our existing model and understand it does not work anymore with the technology we have developed and that a new way of working is an essential next step in the evolutionary process.
Ian Nimmo is president of User Centered Design Services Inc. and an expert on human factors in process control operations.
