Formalize Your Incident Response Plan
A plan that no one knows about is worthless. Communicate the existence of the plan and how it works to everyone who could possible be involved in an incident response team. Make sure that management knows about the plan, how incidents will be handled and how they will be kept up to date in the event of an incident.
Expect The Plan To Be Ignored
Responses to most incidents are studies in controlled chaos. Don’t expect the people who have the most to lose to quietly sit down and review what the Incident Response Plan says before they take action. General Eisenhower, when commenting on the planning for the Normandy Invasion in World War II, said “Plans are useless, but planning is essential”.
What he meant was that when the troops hit the beaches the situations they faced were totally different from what they had planned and practiced. But, the fact that they had planned and practiced gave them the knowledge and experience they needed to adapt. The same is true for Incident Response.
Configuration and Patch Management
Configuration management deals with knowing what is supposed to be installed and running on a device and being able, at any point in time, to determine whether the device has the approved configuration. Manual and automatic processes and tools can be used to check the configurations, which is useful to determine if unauthorized changes have been made to a system. Configuration management is also integral to reconstructing a system or machine in the event of a disaster.
Patch management is the process for testing released patches and applying them to appropriate systems. This is difficult in all environments, but particularly so in control system environments. Patches must be tested to ensure they do not introduce unintended problems as well as their interrelationship with other patches, system and application software. Quite often, updates to system or application software must be tested and installed before patches will work properly. Patch management software systems are closely related to configuration management systems in that they can track the installed base of system and application software with a record of patches that are installed, incompatible, required, etc.
Isolation of the control system telecom environment can reduce the risk of unpatched vulnerabilities in operating systems and applications being exploited, but the danger cannot be eliminated. Effective patch management requires extensive test systems that can replicate every configuration in the control system environment, and implementing patches as quickly as is prudent.
While continuous or at least regular monitoring of systems for infections with malicious code is a needed precaution, it is counter productive to implement a security tool or procedure that prevents a system from fulfilling its business objective. Unfortunately, systems that must complete transactions in a small, discrete portion of time are quite often impacted by anti-virus software.
Usually, restricting the anti-virus software from running on files that are used in real time is a sufficient precaution. With control systems, however, tests must be made that ensure that the system resources consumed by the anti-virus software do not interfere with the process control software.
One of the problems encountered when isolating the control system telecom environment behind a firewall is automatically providing virus definition update files to devices in the control environment. One solution to this is to place a server containing tested updates in the control system DMZ and configuring anti-virus software in the control environment to retrieve updates from this DMZ server.
Log File Analysis
Many of the devices in the control system environment do not support log file creation. But, since most of these systems do not require any type of authentication or authorization, there really isn’t much to log. However, the control systems that use Windows or different variations of UNIX do support logging of security events and these should be utilized.
What to log really should be determined by the security analysts at each company. But, most agree that at a minimum failed logon attempts and failed attempts to access files should be logged. The major problem common to most companies is not keeping the logs long enough and not analyzing them.
One of the benefits that logs can provide is reconstructing what happened in the event of an incident. Thought should be given to the types of information that might be needed and the possible time lag between the time an incident occurs and when it is discovered. Log files that are maintained for only a week and that contain the bare minimum information may not be worth maintaining.
Regular review of the logs is essential. Information may be gleaned that is not detected by other tools such as NIDS. However, since log files are usually very large and there are so many systems, a company should consider the value of purchasing a log file collection and analysis tool to assist the system administrators and security analysts.
Host Intrusion Detection/Prevention (HIDS)
HIDS technology is excellent for detecting unauthorized activity at the server. However, HIDS agents usually consume approximately 5% of a server’s resources. Before implementing HIDS, it is recommended that extensive testing be performed to ensure that the HIDS agent does not interfere with control functionality. Also, many of the events of most interest – attempts at unauthorised access and failed logon attempts – can be detected using NIDS.
Network Intrusion Detection/Prevention (NIDS)
NIDS can be considered a ‘passive’ control in the sense that the agents run on dedicated devices which analyse data packets that flow through the network without introducing measurable latency. Many new intrusion prevention devices have anomaly detection capabilities. Since the network traffic in a control environment is basically static (there may be more or less of it, but the types of traffic and network behaviour rarely changes) NIDS with anomaly detection capabilities may be ideal. Initial tests by several NIDS suppliers and control system vendors are confirming this assumption.
There is no “silver bullet” tool or technique for securing any computing system. No matter what steps you take, vulnerabilities will still exist. However, pursuing a comprehensive security program that is constantly monitored, updated and reviewed by third parties does constitute due diligence and will keep you as secure as possible.
Jay Abshier, CBCP CISSP, is semi-retired from ChevronTexaco and can be reached at firstname.lastname@example.org.