With 348,000 data streams from 50,000 miles of pipelines, 24 natural gas processing plants and six offshore oil platforms, as well as numerous storage and supporting facilities, Ted Porter, process control supervisor, Enterprise Products, has good reason to be sure his company's data flows continuously and accurately. Speaking to attendees at HUG 2014 in San Antonio, Porter says it all starts at the discrete datum level.
"Before you have data, you have datum—one piece of data," Porter said, referring to the seven-layer dynamic scientific method model (DSMM), where each layer depends on the ones before it:
- Datum: one value;
- Data: many values in time;
- Information: observation of data;
- Knowledge: assimilation of information;
- Understanding: when knowledge allows prediction;
- Wisdom: when validation of prediction leads to action;
"This all falls apart and the wrong action can be taken if the underlying data is faulty," Porter said. "Integrity matters."
Closed-loop control may require all seven layers to develop a model, come up with a control strategy and determine appropriate tuning parameters. "Once it's set up, a closed loop repeats only layers one, two and seven," Porter said, "But faulty or incomplete data when setting up a model initially or when trying to control afterwards, will result in poor control loop performance."
For example, self-tuning algorithms can work well when enough dynamic response is present, but during times of relatively steady state, they can detune a loop to where it will not react fast enough during upsets.
"Without human intervention to gather information, gain knowledge, come to an understanding, develop wisdom and take the appropriate action, automatic control can result in garbage in, garbage out," Porter said.
Don't Make Assumptions
Porter described a case where Process Engineering requested tuning of a level control because it was fluctuating more than desired. The tuning parameters showed proportional and integral control action. "The DCS, although limited to a couple months of history, showed that the fluctuations had always been more than desired," Porter said. "However, the corporate historian had more history and showed a marked rise in level fluctuations around six months previous." The log book showed the fluctuation began when a failed valve positioner had been removed and not replaced.
"The lesson here is, conduct a root cause analysis," Porter said. "Determine when the problem started and any events that occurred at the same time. Don't assume."
Data chain delays are another major drag on integrity. "With the consolidation of operations centers and integration of disparate sources, data may travel many miles before reaching the operator," Porter said. Along the way it may go through many transformations:
- From the digital sensor to a digital-to-analog converter in the transmitter;
- From the 4-20 mA signal to an analog-to-digital converter in an I/O card, PLC or DCS server;
- Through a square root extractor in the transmitter, PLC or DCS;
- Through a serial Modbus-to-TCP/IP encapsulator;
- Through an OPC server, a transmit tunneler and possible encoder for passing the OPC data through a firewall;
- Through various common carriers, microwaves and copper-to-fiber converters;
- Through another firewall, a receive tunneler and possible decoder to another OPC server or ("hopefully not, but sometimes") a relational database; or
- To the operator, then perhaps up through another firewall or series of firewalls through a DMZ that may have another server and buffered interface, before passing to the corporate network to a management information system.
"Each digital polling can be round-robin or set-frequency, sometimes with reporting by exception. And you don't expect delays?" Porter asked, then answered: "Expect delays."
Minimize delays by minimizing the number of databases, distances, conversions, disparate technologies and transmit/receive devices, "although sometimes these are desired to boost back up the signal strength," Porter said.
Also prefer active versus passive devices, as these introduce a lag. For long distances, opt for fiber over copper and single-mode over multimode fiber to minimize line loss. "Microwaves and satellites are handy, but a lag will be introduced as the signal is received and then retransmitted back down," Porter added.
Understand Data Chain Breaks
Data chain breaks may be accidental or intentional. "Three storage facilities within a few miles of each other are all running Honeywell Experion and Distributed System Architecture (DSA). In other words, they are all DSA'd together, so each site can see and if need be, control the other site," Porter said. "SCADA tags were moved so they were no longer coming into one storage facility, but instead, coming into another. The tags were also renamed after being moved, and no one down the chain was told.
"The management information system collecting the data from the Experion system at the other storage facility continued to collect data as if nothing had happened. Even with the tag change! DSA is magic!" Newer technology can automatically heal data chain breaks.
Data chains may be intentionally broken as part of a cyber attack. "The Stuxnet computer worm crippled Iran's nuclear enrichment efforts by breaking the data chain between the control system and the centrifuges," Porter said. "It introduced its own data designed to cause premature failure of the centrifuges bearings, and it also broke the data chain to the operators' HMI, presenting to the operator as if the centrifuges were operating normally."
Best practices for cybersecurity include disabling thumb drive ports, blacklisting, white listing, air gapping, firewalls blocking all Microsoft-used ports, stateful inspection of all TCP/IP traffic, dissimilar technology for connected networks (such as Ethernet to token ring), using serial instead of TCP/IP, taking machines down for patching, disabling administrator accounts (or at least the one called "administrator") and separating data from programs to enable locking down the program subdirectories. "All the while, maintain and keep the plant running," Porter said.
Other recommendations for ensuring against cyber attacks include monthly security patch application and reboots, moving behind firewalls, upgrading from Windows XP/Server 2003 to Windows 7/Server 2008 and upgrading service packs.
Many other factors may contribute to data chain breaks, such as lack of fault tolerance, excessive rebroadcasts and interference from high-voltage fields. Data may also be corrupted by high signal-to-noise ratios, the previously mentioned data conversions and range clamps. Sensors and transmitters themselves may contribute problems due to, for example, measurement limitations (such as exceeding range), analyzer lag and instrument unreliability.