First Aid for Process Security

Cooperation and Common Sense    

Besides upgrading components, software and networking, Elwart emphasizes that one of his most powerful security tools is coordinating efforts with his overall corporate IT department. This cooperation may come a little easier at Ergon because Elwart trained as a chemical engineer and worked as a process engineer before migrating over to the computing and software side. Likewise, he adds, his Systems Engineering department has a unique blend of business and controls skills because most of the 9 to 10 staffers previously worked in completely different areas.

"IT people already have network security in mind, but we also make sure they get basic operations training, too," says Elwart. "I want everyone in our department and in IT to understand how our refinery runs 24/7 and why those calls at 2 a.m. are a big deal. This training makes everyone more sensitive to shift changes, scheduled downtimes and production runs. As a result, when control engineers open some ports between the network and control level to get some equipment to work and then leave them open, then IT can follow up and shut down the unused ports." Conversely, some control engineers and operators also get trained in business-level and SCADA security practices.     

Elwart adds that the Systems Engineering and IT departments also jointly confer on scheduling upgrades and other projects to avoid causing production problems. "For example, while other IT departments may add enterprise software over a weekend and create some difficulties, we look at what runs and turnaround times are coming up and try to find the best upgrade times," explains Elwart. "For instance, we won't change Internet Protocol (IP) addresses on a terminal server on Friday, so we can avoid possibly locking out some engineers over the weekend. We also won't change IP addresses on Monday to prevent people from complaining that they didn't see the email sent on Friday. So we usually make any IP address changes sometime between Tuesday and Wednesday."      

Likewise, Ergon also waits until its vendors test new software patches and then installs them during normal downtime. "This is because we've seen more operating interruptions due to spurious shutdowns caused by untested patches than we have from what they're supposed to be protecting us against," says Elwart. "If there's an emergency, we'll call the vendor in to help install a patch."

Elwart reports that other common-sense security procedures used at Ergon include locking its server room so no unauthorized people can plug into the servers inside. Similarly, the Systems Engineering department also changes the default passwords on all its devices, which removes one of the most typical pathways used by outsiders to gain unauthorized access. It also uses several levels of authentication, which means different passwords for different network areas. Ergon uses Cisco firewalls to protect its networks, but it doesn't set them and forget them. It implements well-thought-out rules that will meet Ergon's needs when it configures its firewalls and then constantly updates them.    

"Management at some refineries will  sometimes say, ‘We're nice people. Why would anyone want to attack us?' So they need to be shown that pretty much every industrial facility and application with a network connection has had some kind of probing or potential attack that could be a problem," adds Elwart. "I recently learned about a refinery network that had been under a continuous attack for several months. This was a brute-force attack in which the intruder was trying up to two user names per second to gain access. I believe the application's system was able to show where the attacks were coming from, and the unsophisticated hacker was actually caught."  

Elwart adds that one of the main lessons he's learned from trying to improve Ergon's process security is that people can't be prevented from trying new technologies. "IT sometimes tries to tell people not to use a certain technology, but this doesn't work because many will ignore those instructions," says Elwart. "It's much better to embrace new technologies and then learn to manage them effectively. Much of this comes down to simple education. We always talk about hackers, but many security problems also come from inadvertent mistakes. For example, another refiner found some unexplained photos on a desktop and found they'd been accidentally downloaded to the PC when one of the cleaning crew plugged into a USB port to charge his smart phone. So they educated the cleaning crew not to do that sort of thing, and then also configured their PCs not to perform automatic downloads. People are willing to go along with rules if they're told why those rules are important."

Security Similar to Safety

To increase security awareness and compliance, many process organizations report that it helps their staffs to think about security in the same ways they think about process safety. Some internal security teams use business-type risk assessment (RA) matrices (RAM) based on probability and severity of incidents that are already widely used in their safety efforts. This can help production managers learn that security vulnerabilities can be equivalent in impact to the downtime caused by an over-pressurized valve fire or other accident. So the next time someone brings an unfamiliar laptop into their facility, they'll stop and think, question if it might have a virus and get it checked out.

"Process security is really the second coming of safety," says Rick Kaun, global business manager for industrial cybersecurity at Honeywell Process Solutions (http://hpsweb.honeywell.com). "But, like safety, there's no point at which you're completely secure. It's a constant lifecycle of reviewing, revising and refreshing the three main pillars of process security—technologies and services, policies and procedures, and people. For example, I helped assess a refinery a few years ago, and we found 57 security concerns, and they fixed most of them. However, when we went back two years ago, we found 53 more items, but they were mostly unsophisticated issues. So we'll go back next year, and see what's up."