First Aid for Process Security

Likewise, Suncor Energy Inc. (www.suncor.com) in Calgary, Alberta, Canada, reports that its recent Process Control Network/IT Security Project involved updating its security design, security maintenance and asset configuration management capabilities in its oil sands extraction, upgrading and support facilities. The firm's PCN security maintenance program included improving its testing environment, anti-virus and patch management tools, password and authentication management, security hardening guidelines, and health and security status monitoring.

"Making the case for improving cybersecurity in our budget meant better understanding the importance of our assets and what the lack of protection and security could do to them," says Cliff Pedersen, PE, Suncor's product production processes manager. "We calculated the cost of an unplanned outage if our networks were left as is, and the likely cost was several million dollars. A project of this nature, which spends money, but doesn't ‘make more oil' is difficult to sell. For a process security project to succeed, you need to illustrate the true impact of what you're doing and show the consequences of not doing it."

Brad Hegrat, principal security consultant at Rockwell Automation (www.rockwellautomation.com), adds that, "A basic RA involves inventorying an application's components, categorizing them, rating their criticality, assigning risk profiles and addressing those risks by mitigation, transfer, acceptance and avoidance. It's also crucial to do a comprehensive gap analysis of how a facility's staff sees its security role, what polices are already in place, and then assess technical needs." Besides continually updating passwords, firewalls and patching policies, Hegrat also stresses that users should put PLCs in "run mode," so they won't accept edited changes to their existing configuration or programming.

Security at Salt River

Similar to most multi-regional power plants, Salt River Project's (www.SRPnet.com) Navajo Generation Station (NGS) in Page, Ariz., always had security practices to prevent unauthorized access to its controls and backup policies for any incidents that might occur. And as PCs and software emerged and moved onto the plant floor, SRP kept its control network separated from its IT and business network.

NGS is a coal-fired plant that routinely produces about 2250 megawatts in its three 750-megwatt units and is projected to maintain a typical uptime of 92.5% over the next 10 years. The plant burns low-sulfur, bituminous coal, and its power goes to customers in Arizona, Nevada and California and also drives water pumps throughout the Central Arizona Project.

"However, a few years ago, our plant management saw the writing on the wall that NERC-CIP would eventually designate our plant as a critical asset, and they told us to begin looking towards complying with NERC-CIP Version 3," says Mike Hull, supervisor in NGS's computer control group. "So we put together an SRP security team and drafted a best practices document, so we could move in the right direction, establish a security baseline for all of SRP and have consistency across our organization about back-up strategies, patch management and physical and electronic access control."

Basically, the North American Electric Reliability Corp. (www.nerc.com) and its Critical Infrastructure Protection (CIP) program are requiring electricity producers to draft and implement security policies for their facilities that comply with its NERC-CIP standards. However, even though its security efforts are further along than other industries, critics claim that NERC-CIP allows utilities to define too many assets as non-critical and thereby avoid securing them.   

Second Firewall, Second Network

Hull reports that SRP's and NGS's recipe for security began with identifying all its critical assets and their vulnerabilities and implementing indicated security devices, services and a layered infrastructure needed to protect them. The team found and documented about 3000 applicable assets. "However, this first part of updating technology is the easy part because next we have to do a performance baseline; make sure we're designing our security system and roadmap so it can be rolled into NERC-CIP compliance; and address any new vulnerabilities that show up. For example, NERC-CIP says to define an electronic perimeter and access points and decide what goes on either side. So even though we recently put firewalls around our corporate network, in late 2010, we put a second set of firewalls around our controls network. This was done so we would have defense-in-depth, or multi layers of security, so the plant-level could only push data out to the business side, but not accept any coming back in."

Mike Martinez, principal consultant for the critical infrastructure and security practice at Invensys Operations Management (www.iom.invensys.com), reports that SRP's network infrastructure changes also enabled deployment of centralized anti-virus management and back-up capabilities, improved network monitoring and a remote access- jump server with role-based user authentication for remote access. "We've been working with SRP for many years, so when we learned that these cybersecurity requirements were coming, we were able to help design and deploy a solution that would allow them to meet their exisitng best practices, while being consistent with their future need for a NERC-CIP-compliant program," says Martinez.

While many networks use two layers of firewalls, some are also installing a data-based demilitarized zone (DMZ) between their corporate local area network (LAN) and their control system LAN (Figure 2). This provides an added layer of protection because no communications take place directly from the control system LAN to the business LAN, according to the U.S. Computer Emergency Readiness Team (US-CERT) and its Control System Security Program (CSSP, www.us-cert.gov).
Ernie Rakaczky, program director for control system cyber security at Invensys reports that US-CERT is helping many suppliers find vulnerabilities in their software and is testing security risk mitigation strategies to make sure they work properly.

Besides extra firewalls, NGS's security team also added a second maintenance network built on secondary network cards in its PCs, which is another likely NERC-CIP requirement. So while the original card performs its regular control operations on the plant's dedicated Invensys Foxboro I/A DCS, the second card does back-up, maintenance, patch deployments and other tasks.

This second maintenance network touches all the same points as the process control network at NGS's three generating units, SO2 scrubbers, lake pumps and workstations. The teams also added new physical and electronic access controls and password management for logging onto cyber-related assets. For example, mechanical maintenance staff will no longer be allowed into the control room, but will have to go a clearance office to have clearances issued.

Hull adds that upcoming efforts will take SRP's and NGS's security beyond protecting equipment at one point in time to make it part of their lifecycle and obsolescence planning, too. 

"I think the feeling at NGS now is that we're headed in the right direction on process security," adds Hull. "There's a lot of overhead in maintaining security and compliance. And the process never really ends, so we still have more work to finalize our security infrastructure, implement more secure solutions across the whole NGS facility, update other components, and more thoroughly define our physical and electronic security perimeters. Later on the roadmap, we'll check for vulnerable assets again and see how well we did now."  

Jim Montague is Control's executive editor.