Control system device insecurity is addressed by the Presidential Executive Order but is being ignored at your own peril
You can’t secure control systems and the critical infrastructures without having the right people and processes in place. Consequently, there are two messages in this blog. The first is people - control system experts need to be part of the process. The second is process - compromising the Level 0 process sensors and devices can cause catastrophic damage that Operational Technology (OT) monitoring cannot detect.
Process sensors are the input to all process control and safety systems including those used in the electric grid. They are part of the Engineering/Operational organizations though the organizations responsible for process sensor design, development, installation, and maintenance are generally not part of any cyber security team. Operational Technology (OT) networks assume sensor measurements are secure, authenticated, and correct. OT generally is under the purview of the CISO generally without Engineering/Operations involvement. This gap in governance was identified in my article, “Attention Policymakers: Cybersecurity is more than an IT issue” published in the May/June 2020 issue of PE- the Magazine for Professional Engineers. This governance gap becomes existential when cyber issues affect the lowest levels of the Purdue Reference Model – the process sensors and actuators as they directly affect safety.
A sophisticated hacker will make a cyberattack look like an equipment malfunction. Given that premise, Honeywell’s Sinclair Koelemij May 24, 2020 blog on the OSI PI – ICSA-20-133-02 vulnerability becomes very importnat. Even though the advisory mentions an attack by a “local attacker”, a local attacker can easily be replaced by malware. Consequently, local or remote doesn’t make a difference. As HART-IP has essentially no native cyber security, cyber security capabilities are dependent on the system integrator or end-user. When an attacker gets access to the OSI PI connector, it is possible to inject other commands using HART-IP affecting the field equipment. Commands can result in modifying range, span, engineering units, and/or damping values. Some field devices even allow the low range to be set higher than the high range value. Such a change would effectively reverse the control direction. The situation can be even worse if both the field devices of the Basic Process Control System (BPCS) and the Safety Instrumented System (SIS) are connected to a common system. In this case it becomes possible to launch a simultaneous attack on the BPCS and SIS, potentially crippling both systems at the same time with potential devastating consequences for the production equipment and the safety of personnel. Often these and other critical systems such Computerized Maintenance Management System (CMMS) and Instrument Asset Management System (IAMS) reside at Level 3 of the process control network. Sinclair and I consider these architectures bad practice to expose the field equipment in this way. There should not be a path from Level 3 to Level 0 without a guarantee that only data can be read. In Sinclair’s and my opinion, this architecture poses a high cyber security risk. The joint ISA 84 (process safety)/ISA99 (cyber security) Annex H recognized these cyber issues. Additionally, these devices, including the SISs, are explicitly included in Presidential Executive Order- 13920.
The unique cyber issues associated with control system field devices were not included in the recently released DOE/DHS CISA infographic. There is nothing in the infographic that can be used to determine if hardware backdoors are inside equipment which was the reason for the Presidential Executive Order - https://www.wsj.com/articles/u-s-seizure-of-chinese-built-transformer-raises-specter-of-closer-scrutiny-11590598710?emailToken=3d85bf968355cc917220da50f9b7d496aep+m7M4+8QH3+Ko3zlkEq57f6+qAwWNh+S7eWJxBxQ110rspASHadV75GoMSH+deHpnXopZ3kCaaSbYhEyk9M3l02l8kogo53ANiSKgMJlbbSFL0gm0jPaEE+vIl40E8ih85jsPW6l3xFNjtTkRAg%3D%3D&reflink=article_email_share. Ironically, the one concern with that article is the purpose of this blog – ignoring the sensors. The Wall Street Journal article states: “However, transformers hadn’t typically been seen as products that could be easily isolated and hacked. That is because they don’t contain the software-based control systems that foreign actors could access. They are passive devices that increase or reduce voltages in switchyards, substations and on power poles according to the laws of physics." That statement is misleading. I was informed that at least some of the devices in the compromised transformer had a “manual” load tap changer. However as mentioned above, manual or automatic, the device, in this case the load tap changer, will respond to voltage fluctuations it sees from the sensors. If the sensors have been compromised via the backdoor, the transformer integrity is at risk. Compromising sensor configurations can lead to failures of controllers, voltage regulators (load tap changers), and other control system devices with no apparent indication.
The recently published OSI PI security issue clearly shows there is a need to be careful with how these systems are connected and the consequences if such systems would be breached. Network segments are created to reduce the risk for the most critical parts of the system such as field devices, and though many will say this application is just an interface that only collects data from field instrument for analysis purposes and therefore it does not create a high risk. This assessment will be completely different when it is considered what a threat actor can do when the threat actor (malicious or unintentional) gained access to the server and misuses the functionality available.
The 2017 Triconex cyber security attack in Saudi Arabia was intended to prevent the SISs from operating in order to cause physical damage to the plant. The malware was found when the plant tripped because of the complexity of the triple redundant safety system. The gap in coordination between OT security and safety engineers contributed to allowing the plant to restart with the malware still in the Engineer’s Workstation. However, there has been little written about the DCS that would also have to be compromised. Compromising the process sensors feeding the SIS and BPCS could have led to the same goal without the complexity of compromising both the BPCS and SIS controller logic and the operator displays. As mentioned, SISs were explicitly identified in the Presidential Executive Order.
There is another fascinating aspect to this story and it involves the Pacific Gas & Electric Company (PG&E). As readers may be aware, PG&E is a convicted felon for causing the 2010 San Bruno natural gas pipeline explosion (a control system cyber incident). PG&E is in bankruptcy because of the wildfires attributed to their equipment. September 17th,, 2019 I had a meeting with PG&E’s cyber security lead and legal council – a meeting arranged by PG&E’s Chairman of the Board which was to have been about the need for control system engineering participation in PG&E’s control system cyber security program. However, there was no control system attendance (see above PE article). Fast forward to the beginning of the year where a contractor found thousands of process sensors connected directly to PG&E’s corporate network despite PG&E’s continuing declaration of safety being important which is a major safety and security compromise - https://www.controlglobal.com/blogs/unfettered/the-nerc-cips-are-not-designed-to-keep-the-electric-industry-safe-from-cyber-incidents-or-attacks/). What will it take to change (create) PG&E’s safety culture before they are allowed to emerge from bankruptcy? PG&E is not the only culprit. I talked to another utility about the Executive Order. The answer was we have no budget. There was a nation-state cyber attack against the bulk electric system and a resultant Presidential Executive Order. Wouldn’t you think budget would be made available?
I have been writing about the lack of process sensor cyber security for years. This is an issue that has either been ignored or demeaned by many in the US cyber security community – see https://www.controlglobal.com/blogs/unfettered/sensors-and-sensibility-dragos-and-other-ot-experts-lack-expertise-on-process-sensors . However, the British thought it was important https://www.controlglobal.com/blogs/unfettered/the-british-government-thinks-process-sensor-cyber-issues-are-real-what-about-everyone-else
There are some technologies in various stages of development as well as policies and procedures that can address some of the critical issues inherent with compromised or counterfeit equipment. But, as my PE article states, engineering must be involved.
The Chinese and Russians have cyber attacked our critical infrastructures by compromising the weaknesses in people and processes. There is now a Presidential Executive Order in place that can help address these weaknesses. When will we learn?
Joe Weiss
