Cyber security for the electric sector

By Dale Peterson, SecureSystems Insider Contributor.
and Doug Howard, Counterpane Internet Security 

MOST COMPUTER networks facilitate the flow of information. But in the electric industry, special computer networks control the process of generating, transmitting and distributing power. Distributed Control Systems (DCS) allow a small number of operators to control a power plant. Supervisory Control and Data Acquisition (SCADA) systems control and monitor the transmission and distribution of power across a wide area from a control center. These systems control a process that has serious human safety implications and are essential to the critical infrastructure.

Control systems have many unique aspects that aren’t found in a corporate network, from protocols and equipment used to control the physical process, commonly via measurement and actuation, to special performance and availability requirements to prevent downtime or delay. But these same control systems also have much in common with corporate computer systems that are constantly under attack. Control systems are increasingly connected to the enterprise network, which is connected to the Internet. Key components of control systems run on Windows and Unix operating systems.

Forward-thinking industry experts were worried about a cyber attack on control systems prior to the year 2000, but this was not widely embraced as a real concern.

All this changed after Sept. 11, 2001.

The industry was faced with the reality that a sophisticated and dedicated adversary wanted to damage the US and other countries. The ease at which a cyber terrorist could knock out large portions of the electric grid became a real concern.

The Northeast blackout in August 2003 was a second wakeup call because computers and applications failed to work properly. The investigation found no evidence of a cyber attack causing the blackout, but the incident highlighted the potential for a future attack.

These serious threats, along with the ubiquitous worms, viruses and general hacking faced by any computer user, underlined the need to insure appropriate cyber security measures are in place to protect the DCS and SCADA systems essential to the generation, transmission and distribution of electricity.

NERC Steps In
The North American Electric Reliability Council (NERC), with its stated mission of ensuring the reliability and security of the bulk electric system, was the logical choice for regulating cyber security for the electric sector.

The need was so critical that NERC did not follow its typical process for developing a standard. Instead, Urgent Action Standard 1200 – Cyber Security was issued in August 2003 and renewed in August 2004.

During this two-year period, NERC worked on a longerterm solution, now split into eight critical infrastructure protection (CIP) standards:

 

• CIP-002 Critical Cyber Assets
• CIP-003 Security Management Controls
• CIP-004 Personnel and Training
• CIP-005 Electronic Security
• CIP-006 Physical Security
• CIP-007 Systems Security Management
• CIP-008 Incident Reporting and Response Planning
• CIP-009 Recovery Plans

While implementation is likely to vary a great deal by sector, the underlying requirements could be used in any industry with little modification. The NERC CIP requirements are similar to banking, e-commerce, health care or government best practices. They require:

• A cyber security policy;
• Employee security training and awareness;
• Disabling unused network ports/services to limit what can be attacked;
• Strong passwords (a mix of character types of sufficient length that would be hard for a person or program to guess) for user authentication; and
• Monitoring the security perimeter and critical assets for attacks.

Measurement and accountability are key features of the CIP standards. Each standard includes the audit requirements to achieve compliance and requires a “senior management officer’s approval,” which will certainly help achieve management buy-in to cyber security.

NERC chose to be very general in the requirements, rather than state how to meet each requirement. It’s certainly easier to achieve consensus on general requirements and meeting them is bound to improve cyber security to some degree. But a lack of specifics also means a company could be compliant from an audit standpoint without necessarily achieving the intended goal: security.

The standards and related audit requirements apply only to “bulk electric systems,” and most organizations already know if they are subject to NERC standards. But electric systems that don’t fall under NERC’s “bulk electric system” definition would have a cyber security program if they voluntarily comply with these standards.

To take this a step further, other critical infrastructure fields, such as chemicals, oil and gas and water, could use these standards until a more applicable one is developed for their industry.

Complying with NERC CIP
All organizations fall into one of five stages in the evolution toward compliance with the NERC CIP or other compliance mandates such as Sarbanes-Oxley, ACC’s Responsible Care or HIPAA.

Stage I: No clear understanding of the organization’s risks and liabilities in relation to cyber security.

Stage II: General understanding of risks and liabilities, but cyber security as a program is ad hoc and purely reactive.

Stage III: Cyber security is defined as a program with a clear understanding of risks and liabilities somewhere within the organization, but is highly dependent on individuals.

Stage IV: The cyber security program is implemented as a cross-functional process and generally understood throughout the organization, with minimal dependence on individuals for its perpetuation.

Stage V: The cyber security process is measured by the organization in terms of the human and technology costs, incident response times and standardized reporting involved in mitigating critical vulnerabilities and responding to attacks as they happen.

As a rule, organizations rarely jump over stages in implementing cyber security programs without help from external supliers. A common mistake made by organizations is to aim for Stage IV when starting at Stage I or to aim for Stage V when starting from Stage II. Implementing a robust policy, coordinated with a process enabled by predictable, automated technologies and reporting mechanisms, takes time internally.

Yet the NERC CIP compliance schedule will require just such a jump for many electric systems. The CIP standards are scheduled to become effective on Oct. 1 and electric systems must comply, in varying degrees, as soon as the first quarter of 2006. Given the current level of comments on the drafts, this schedule may slide but probably not more than a couple of quarters.

Fortunately security technologies and processes have evolved considerably over the last five years. Vendors offer a range of tools, technologies and fully automated outsourced processes that can speed the CIP compliance effort. These products and services need to be factored into the cost estimates.

The cost of complying with NERC CIP varies significantly depending on which stage the organization is at. The size of the organization also matters, but given the fixed costs associated with continuous monitoring of the network by trained IT and SCADA Security personnel, the size matters less than the stage.

Key components of cost of complying with NERC CIP include:

• Policy and procedures development and implementation;
• Dedicated security personnel, 24x7x365; and
• Intrusion detection and monitoring.

Policy and Procedures Development and Implementation
Security policies and procedures provide the enabling “glue” to bind a sustainable, scalable security program. Operational procedures span change control, configuration management, patch management and back up and recovery in relation to overall information security policies.

For organizations at Stages I or II, the consulting fees alone to develop and implement robust policies and procedures start at $50,000 for a small utility operation and $500,000 for large utilities. Stage III organizations can expect a 30 percent lower entry point given the running start they have toward the definition stage of the work. Their expenditure is required to roll out effective training programs with demonstrated results. Stage IV and V organizations likely will not require outside services to continue improving themselves in relation to NERC CIP and other mandates.

Dedicated Security Personnel, 24x7x365
At any stage, personnel costs generally dwarf technology costs, since companies must staff an incident response process with dedicated personnel, 24x7x365. Typically, personnel costs represent 70 percent of ongoing resource requirements. Furthermore, the expertise required to staff these positions is in relatively short supply given the specialized skills associated with installing, managing and monitoring network security systems. Outsourced managed security service providers (MSSPs) have emerged as a result of the opportunity to aggregate expertise for application across hundreds of networks from security operations centers designed to deliver these services.

The average cost of a fully loaded, full-time-equivalent trained security professional is $120,000 per year, assuming a base salary for a certified information systems security professional of about $85,960, plus overhead costs of approximately 40 percent over base. Assuming a minimum of three FTEs are required to staff a continuous incident response process, $360,000 per year or $30,000 per month is the minimum entry point for any size organization to fully comply with not just NERC CIP but Sarbanes-Oxley and HIPAA.

These security professionals must review anywhere from 2,000 to 3,000 alerts per year, or 5 to 10 a day, if network security devices are properly tuned to reduce the number of false positives they generate. When devices are not properly tuned, the volume of alerts is overwhelming. For instance, organizations at Stage I, II or III often install intrusion detection systems (IDS) but tune them into a state of irrelevance for lack of time.

Intrusion Detection and Monitoring
CIP-005 requires all bulk electric systems to have a 24- hour intrusion detection capability to detect intrusions and intrusion attempts at the electronic security perimeter and on critical cyber assets. For utility companies, specific IDS signatures have been written with funding from the Department of Homeland Security Advanced Research Projects Agency (HSARPA), to identify attacks embedded in SCADA and DCS protocols. These signatures are primarily focused on the MODBUS TCP and DNP3 protocols, which are widely used in the electric industry. A single IDS sensor can identify attacks on Microsoft operating systems and SCADA field devices such as IEDs and PLCs.

Organizations will require a minimum of one IDS, and larger enterprises will need up to four. Costs start at $40,000 to $50,000 for typical enterprise-grade vendor-supported products.

Monitoring technology used internally by an organization, typically referred to as Security Information Management systems (SIMs), ranges widely in cost. But they typically start at $100,000, including systems integration fees, for any organization large enough to deploy the technology. According to the Gartner Group, the cost and complexity of using SIM tools put them out of reach for all but the top 20 percent of the Fortune 1000. SIMs collect syslog events, Windows event logs, SNMP traps, firewall logs and other information from all the security devices in the organization, store that information in a common database, analyze it and present it in a format that is easier for security specialists to interpret.

All or part of the intrusion detection and monitoring can be outsourced. For example, the management and monitoring of an IDS sensor can be outsourced for $750 to $1,500 per month. Management and monitoring of the protection devices, such as firewalls, and monitoring of key servers that are designated as critical cyber assets can also be outsourced. Most of the outsourced services allow full visibility of the monitored information so internal resources can be as involved as they need or want to be.

Vendors provide reporting tools for each set of products monitored, such as firewalls and IDS devices. Alternatively, reports are provided as a standard part of MSSP services. Most importantly, reports must provide an uninterrupted audit trail for review by internal and external auditors.

Conclusion
The NERC CIP standards will be in force shortly. The requirements are very similar to best practices found in other industries, but the implementation will need to take into account the critical availability and performance requirements in a DCS or SCADA system. Given the approaching deadline, bulk electric systems will need to find the right mix of products and services to implement an effective and compliant cyber security program in the available timeframe. They also need to keep an eye on the long-term cost implications of these decisions.

---

About the Authors

Dale Peterson, CISSP, is director of network security practice at Digital Bond, Inc. Contact him at peterson@digitalbond.com or call 954/384-7049. For more on the company, go to www.digitalbond.com. Doug Howard is vice president of service delivery at Counterpane Internet Security. He can be reached at dhoward@counterpane.com or call 703/227-5940. For more on the company, go to www.counterpane.com.