Lessons learned from HMI upgrades

By Bridget Fitzpatrick, Mustang Engineering

Control systems have changed a great deal over the last 20 years, and control system upgrades are commonplace. Most seasoned automation engineers have been through at least one major upgrade, and the more highly experienced often have battle scars from multiple generations of migrations. But it is only in recent migrations that the human machine interface (HMI) has become recognized as an important role in these projects.

As control systems have become more open and more powerful, the HMI has become more complex with an ever-increasing set of tools to aid the operator. Operators used to be able to see their span of control with a stroll along the panel board or through a handful of process displays.

But as the span of control widens and the numbers of displays grow, the HMI must be redesigned to manage the flow of information and effectively become the funnel through which all available information must pass to reach the operator.

Data Sources and Managing the Funnel
As the control system has advanced over the last twenty years or so, the wealth of information available for delivery to the operator within the control console has grown significantly.

THE HMI PROVIDES DATA TO THE OPERATOR Operating Procedures Safe Operating Limits Daily Orders Quslity Information Tank Information Warehouose Information Real-time Sales Orders Process Data Technology Manual Maintenance History Lab Information Work Orders Process Models Batch Record Operating Log Environmental Permit Information Blind List

If all this information was continuously displayed, the important information would be lost in the clutter. There is simply too much information to monitor it all. There are technical limits as well and, if the information delivery is poorly engineered, then the HMI can be confusing, slow to respond, and, in worst cases, will fail to respond correctly or at all. But if the information to display is intelligently selected and engineered, the HMI can be a dynamic tool that enables the operator to effectively manage the process through all modes of operation.State Detection and Automated Information DisplayIncreasingly, systems are designed to automatically manage the information display based on conditions detected in the process or in response to operator interaction. Condition or process state detection is a field with heavy research and development in recent years. Implementing state detection and an HMI that is dynamic based on this state detection needs to be done with caution.It is important to understand the mental models that operators use to operatore and provide a bridge in the migration from the old view to the new design. Designing that bridge is not always straightforward. I once worked on an HMI upgrade for an application with two process trains, one generally in operation and one in regeneration. The first pass HMI showed mostly only the vessels and equipment for the parts of the process in active use for operation or regeneration respectively, though placed in the same relative position as a reference (a safety review identified what instrumentation needed to be shown in each case, an important consideration in dynamic HMIs).We had the new and the old HMI running in parallel and the operators refused to use the new HMI. We went back and put in the rest of the process and showed it in a dull color and static manner and the operators started using the new HMI. They noted that they understood the process better having only the important instrumentation shown. We had not originally shown the idled vessels, thinking them a distraction, but they were critical to operator understanding.Similarly, implementing an HMI that changes significant parts of the interface in response to routine operator mouse-clicks should be undertaken with caution. Most HMI’s automatically change the basic interaction zone (commonly called the change zone or faceplate) as the operator mouse-clicks through the process. Others change the standard trend windows. It is certainly also possible also to change such things as the overview and detail level displays and to provide information on maintenance records, standard operating procedures, environmental permit compliance, related daily orders, and logbook entries to match the subject of the current mouse-click of the operator.Some level of this can be a very effective tool for the operator, but it all carries an ongoing system loading and a maintenance burden. Ultimately, if you distract the operator from what he or she is trying to control with constantly changing information, then you hinder rather than help operations.But do not throw out the idea. Certainly, operators must have areas where they can position the current day’s problems and keep them under relatively constant surveillance. This surveillance is an important method of operation and should not be removed. You can help the operator by putting related information in secondary locations, so that it is always there when he or she needs it at a glance and do this to the level that you have space. If the screen space is limited it is better, in my opinion, to provide this related information on a right-click menu and leave the operator to operate, knowing that information is a right-click away.One of the danger zones for the HMI is that of creeping elegance. It can be very easy to let the ability to do something overcome the reasoning of what the user really needs in the HMI. Ensure that you have a clear definition and strong consensus before you proceed in automating display changes. Becoming more functional and more elegant on the delivery of on -demand data has far less fewer risks and more likely rewards. The list of possible information is often only limited by your imagination, tempered by what is worth the investment in infrastructure, development time and system maintenance.A Word on Project ApproachesThere are a variety of approaches to the HMI Upgrade project. The most common are:

• Replacement in kind
• Upgrade to a new basic HMI standard
• Upgrade with new tools.

One might think, understandably, that the replacement in-kind project might be the most straightforward and less prone to scope and schedule issues. However, flaws in the existing system can make this difficult.

Upgrading to a new basic HMI standard is probably the easiest project type to manage. The new standard provides clear documentation and a clear scope of work. Upgrading with new tools is also likely to be simpler than replacement in-kind.

However, management of the pace of change from the old to the new advanced graphics can be a thorny constraint to manage. These advanced projects are achievable and well worth the effort, but they often must be managed with a phased approach. And, of course, tools that are advanced enough to rely on the latest and greatest software have a larger potential exposure for cost and schedule with more features to test and more bugs to resolve.

Dealing with Reality
Perhaps the hardest issue to deal with on any HMI project is the requirement to live in reality, with all its associated flaws.

One flaw is that the P&IDs and existing displays all have errors. No matter how often they have been walked down, or how strongly everyone thinks otherwise, all drawings have flaws or at least items that will raise questions. The most common questions include such things as instrument placement (for example, is the pressure before or after the manual block valve that we are showing on the graphic?) or naming conventions (something as simple as the P&ID calls the material Resin and the existing graphic calls it Plastic).

In most cases, sometimes the P&IDs and the existing graphic are correct. Or both are correct and it is a matter of determining what naming conventions will be used on the displays. It is surprisingly common to find instruments, or even controllers, drawn incorrectly on an operating graphic. Once identified, long-standing “control” issues are surprisingly corrected. It is important to keep asking questions until any discrepancies are resolved. Any corrections at a minimum are part of the continual improvement of the documentation and occasionally you will make a real improvement in process understanding.

Another problem is that existing displays have Band-Aids. Workarounds for problems with instrumentation or procedures are present in most systems. In one upgrade, I ran across an odd script in the stop/start for the fan on an incinerator. In addition to the expected hand controller for the stop and start, there was a second relay being switched off and on. We searched for a loop file and traced out the wires on the panel drawings. We could not find anything definitive and the fan was secured, so we took the second relay out and tested the stop/start. The incinerator fan became self-starting. No matter what we did, the fan kept restarting. We kept looking and found some old electrical prints for the antiquated VFC (variable frequency controller) and it turned out that a time-delay relay had been wired incorrectly and had never worked. So, the HMI had been scripted to fix the problem.

The best way to manage the flaws in the reality of the system is to follow a structured approach. Mustang Engineering employs a ten step process (see below) that manages reality, as well as budget and schedule.

MUSTANG ENGINEERING HMI PROCESS

A Ten Step Process That Manages Reality

1.    P&IDs

2.    Database Reconciliation

3.    HMI Design Guideline

4.    Navigation and Alarm Hierarchy

5.    Operator Task and SOP Review

6.    Schematics

7.    Build System

8.    Groups & Trends & Third-party Interfaces

9.    Functional Acceptance Test

10.  Final Documentations

Managing reality is as hard as managing the process.


Mental Models
Operators and engineers run a process based on their internal mental models of the process. They monitor the process and check the ongoing data against their models and respond as soon as something makes sense to them. Theories will be discarded only after several new pieces of data do not fit the model. All people do not have the same mental image or model of the process. So it is important to involve more than one operator and more than one engineer and make a composite image from all of the input. As you build the HMI, get feedback from the wider audience to sanity check your design.

It is important to understand that every “great idea” you have will not work. It is important to remember that the operator is the ultimate user for the process graphic and it should most closely match the mental model that you discern from their input. But there should also be control displays that provide enough detail for the engineer to troubleshoot the detailed control design. These should match the engineer’s mental model, but still be useable by the operator. Ultimately, keep working on it until it also makes sense to you. You are not an operator or engineer on the process, but you are a credible professional, and it should make sense to you as well. If it does not make sense, there are likely some minor imperfections that might eventually become important.

Remember that you are doing more than drawing pretty pictures. Engineering the HMI to account for human factors and enable understanding across every mode of operation is a challenge. The HMI is the funnel through which all information must pass and to be processed by the operator. Designing this system well can make a huge impact on both safety and profitability.

Lessons Learned

1. Think of the HMI as the funnel that filters information. Make it a well-designed filter.
2. Dramatic changes in how the process is shown can be confusing. Make a bridge from the old to the new.
3. Just because you can automate does not mean you should. 
4. Scope can be easier to manage when upgrading to a defined new standard.
5. Even if there is no visibledoensn’tdoesn’t appear to be a reasonable explanation for something, it has a reason and you had better keep looking until it is understood.
6. Everyone does not think alike. Design the process displays for the operator and the control displays for the engineers. Keep at it until it all makes sense to you as well.
7. Inconsistent error display will lead to confusion and tension.

About the Author

Bridget Fitzpatrick is Principal Technical Consultant at Mustang Engineering, Houston, TX, and may be contacted at[email protected].