Marty: I am commissioning a model-predictive controller (MPC) remotely from home. I could be doing it from Navarre Beach. I can analyze and improve an advanced process control system (APC) and take a MPC in and out of service by remote access. I love the immediate knowledge and results of continuous processes. With batch processes, especially bioreactors, you have to wait 10 days for the batch to complete, plus a few more days for lab results on the product. In the separations business you can make changes and see the results right away.
Stan: How do you avoid the perception of APC as a threat?
Marty: You don't force something down workers' throats. You reassure them the higher level of automation will not take away their job, but will free them up to help solve problems in the field. They can leave the console. Operations can focus on dealing with abnormal conditions. The company holds them accountable. The loss of purity is an incident and depending on the severity and customer impact, goes up the chain of command.
Stan: How do you help operations gain confidence in APC?
Marty: We had to prove what APC can do, be there and reassure operations they are not getting the typical APC taillight guarantee (we are done when you see my taillight so we can jump to the next sexy project). I promised in 2008 we would not let go. We could not guarantee how fast or how a problem would be fixed, but did guarantee we would provide the support necessary to solve the problem. MPC is now the standard, and APC is viewed as essential.
Greg: How do you help operations when things go wrong?
Marty: We can support our plants anytime of the day anywhere (home, airport, coffee shop, etc. provided there is an Internet connection). When operations are having problems, your virtual presence in the control room is worth its weight in gold. When the process has changed or is in a zone never seen before, we need to know this before operations takes the full brunt of the impact. Often the explanation from operations is really not the root cause; rather it is a symptom of some larger problem. Since we often do not speak the same language, seeing the same thing makes a world of difference. We also do primary and refresher training remotely on their systems with newer technology. My group can service over 40 plants, adding about five plants a year.
Greg: On a personal level, what has remote access meant to your group?
Marty: We have cut travel time by 40% for our group. We are decreasing burn-out and saving marriages! You have to have a very understanding spouse to be gone 70% of the time. New employees spend more time in the plants initially, but graduate to remote access as soon as it makes sense and the proper relationships with the plant personnel have been built. It takes credibility and trust from operations to make changes to the system without being present. We can hire and retain talent with this plan.
Stan: What have you done to prevent hardware from holding you back?
Marty: Currently we have sourced computers with a set specification (motherboard, memory, drivers, operating system, etc.) for APC use. We have spare replacement computers should a plant APC computer fail. We maintain image backup of the APC computers and are able to reimage a replacement computer for a quick installation. However, we are now in the midst of going to a VMware solution. We will be able to run as many virtual machines as we need. We would not be limited by RAM and processor capacity. We would have high reliability by redundancy. It is expected that hardware failures will be less common. The hardware becomes less critical. Outside of remote access, virtual computing has the potential of making me the happiest in the last two years. I will no longer have to take care of the computers in the plants.
Greg: What kind of people do you have in your group?
Marty: I am fortunate to have three silver foxes. The youngest is 50. These guys carry a lot of weight in the plants. They have extensive regulatory control and practical process besides APC knowledge. There is one guy in his 60s. He is able to break down complexity to simple things in many buckets and pours them together to reformulate the problem. I told him when he retires, I go too. We are so lucky we have people who love what they are doing.
Greg: How do you bring new engineers up to speed?
Marty: To be effective quickly, prospective hires must have a solid foundation in regulatory control and a willingness to learn the process right away. We can teach what they need to know about APC. New engineers gain the practical process knowledge they need by sitting with operations, and often times it takes many months. They ask a lot of questions and show they care. They learn what the operators do when they have an upset and why. They find the best handle, especially when a quick correction is needed. The most obvious handle is not always the best.
Stan: How does this knowledge affect MPC design?
Marty: You want a steady-state controller to properly handle changes (robustness) and abnormal situations. Otherwise the operator says the MPC is not working. The solution may need to be put in advanced regulatory control. Even if you could get a model and handle the problem in the MPC, with a 1-minute execution time, a few missed calculations, and the MPC may end up taking too long to make the full correction. Also some deviations you need to just leave alone and not chase. High-speed disturbances that exhibit some level of self-correction are better left alone because chasing them may create a bigger upset than just ignoring them.
Greg: What about problems with the final control elements?
Marty: Even if you have good valves, you can get into oscillations without a good positioner. Sometimes there is a burst of oscillations that may go away. I despise software solutions to hardware problems, but we have used a dead zone to ignore oscillations. You need to get at the root cause. For example, we found erratic guide vane behavior resulting in the compressor not being able to keep up with the flow controller. You need to know the fix and how much time to repair to plan the downtime work. Shutdowns are scheduled based on pricing. When it is hot, you may shut down for a few hours or for a few days.
Greg: A vane, damper or control valve that does not respond fast enough may develop a burst of oscillations. The burst may only occur for large upsets or large changes in setpoint. We use external-reset feedback (dynamic reset limiting) in a PID to prevent the controller output from changing faster than the secondary loop or final control element can respond.
Stan: What is your typical MPC?
Marty: The average MPC has six to eight manipulated variables (MV) and 12 to 14 controlled variables (CV). Some of these are for minimization and maximization rather than control at a target. For control at target, we have greater move penalties when at steady state. Our MPC execution is once per minute. We push as much as possible to the regulatory control. For poor measurements (e.g., flowmeters with too much noise or not enough rangeability), we write directly to the control valve.
Greg: What are you doing for flexible manufacturing?
Marty: The MPC does an excellent job of ramping plants automatically based upon customer or pipeline demands. The MPC enforces the mass balance which prevents chasing analyzer signals during the transition. When at the new steady state, the MPC lets the analyzer do its thing unless the analyzer is trying to put the process too far away from the mass balance. There a few analyzers, such as purity to customer, that must take over. When a plant is being ramped, the MPC models do not always hold and confusion reigns. We want newer analyzer technology to help us. The older analyzers typically don't have symmetrical loading, causing the MPC to over correct if model differences are not accounted for.
Stan: What are you doing for optimization?
Marty: Corporate can dial in a new setpoint for the MPC. We are getting into real-time optimization (RTO). We love customers that tie into our pipeline. When the pipeline calls in with an increase in demand, the most efficient plant ramps up. For decrease in demand, the least efficient plant ramps down. The MPC can do this in 1½ hours versus the 3 hours by operators. If the ramp is too slow, efficiency is lost. If the ramp is too fast, you could shut down a user. Everything is energy-intensive since everything is compressing.
Greg: How can you tell if an MPC needs improvement?
Marty: We have a key performance indicator (KPI) grading system. An in-house solution was developed by R&D using innovative MPC statistics. We are able to see ahead of time if MPC performance is deteriorating from rising prediction errors. The statistical measure takes into account whether the CV is for maximization, minimization or control. By considering limits and targets of these CV, we can keep false alarms at a minimum.
"Top Ten Reasons to do APC from your Home"
(10) Can immediately implement an inspiration.
(9) Can watch the ball game on one of your screens.
(8) Get to wear shorts and sandals.
(7) Get to listen to music rather than alarms.
(6) Lose weight from not eating doughnuts.
(5) Can BBQ while solving control problem.
(4) No more lonely nights and meals.
(3) Your kids start to recognize you.
(2) Your kids want to become automation engineers.
(1) Your spouse starts to offer you advanced process control.
