Information Technology | Control Global

Information sharing of both traditional IT networks and control system OT (Operational Technology) network device and system cyber vulnerabilities are done by the private sector and the Federal and State governments. There are specific criteria for evaluating the risk level of the vulnerabilities. However, information sharing on unintentional control system cyber incidents and malicious cyberattacks is close to non-existent (for example, see https://www.controlglobal.com/blogs/unfettered/information-sharing-on-control-system-cyber-incidents-is-not-working-and-that-can-be-deadly). The challenge is when the organization doesn’t collect data and track control system cyber incidents to assess and understand what has actually happened, how can the organization develop appropriate control system cyber monitoring, mitigation, or training? Government and industry are moving full throttle on protection from those OT network-based events they expect to occur and can monitor. Unfortunately, when something happens that doesn’t fit those parameters, there is no process in place to identify the problem or vulnerability and take corrective actions - https://www.controlglobal.com/blogs/unfettered/dont-overlook-the-most-consequential-control-system-cyber-events-of-2020. Because there is a lack of control system cyber forensics and training for cyber incidents that occur at the analog or serial level, these incidents are generally not identified as being cyber-related even though they can, and have been, catastrophic and deadly. As an example, my control system cyber incident database contains more than 1,300 incidents that have killed more than 1,500 and caused more than $70Billion in direct damage.

The Chinese-made transformer seizure has raised unanswered questions both domestically and internationally. As an example, I have been asked by senior government officials from close US allies for information about the status of the Chinese-made transformer problem addressed by Presidential Executive Order 13920. There have been numerous similar questions from US utilities. Because of the lack of information-sharing there continues to be skepticism as to whether the transformer case is real. Examples that encourage the skepticism include the Idaho National Laboratory’s (INL) Sarah Freeman’s RSA presentation that addressed Chinese-made Lenovo PCs with hardware backdoors but did not address the Chinese transformers (see above referenced blog), the May 2020 SANS report that debunked the Chinese transformer disclosure (https://www.controlglobal.com/blogs/unfettered/presidential-executive-order-13920-was-not-due-to-a-malware-event-recent-and-upcoming-events-will-discuss-the-event/), recent Linked-In discussions, and the lack of either the NERC Lessons Learned process or the Electricity Information Sharing and Analysis Center (E-ISAC) addressing the Chinese transformer issue. In particular, the SANS report pointed to a lack of direct confirmation of concerns about a hardware vulnerability in the transformer supply chain. Yet, at least one governmental organization trying to understand the origins of the Presidential Executive Order 13920 independently came up with the same conclusions as mine.

Specific to the transformer case, after a detailed transmission planning study by a domestic utility that was approved by their Board, the utility ordered large Chinese-made transformers because procurement requirements forced them to go low-cost. In this case, it meant procuring Chinese-made transformers with associated Chinese support even though the utility was concerned about the potential for what could, and did, actually transpire. After finding extraneous electronics in the Chinese-made transformer during site acceptance testing, the next large Chinese-made transformer was intercepted by the Department of Energy (DOE) and sent to the Sandia National Laboratory (SNL) for detailed examination. The May 27, 2020 Wall Street Journal article describing the seizure of the Chinese-made transformer (https://www.wsj.com/articles/u-s-seizure-of-chinese-built-transformer-raises-specter-of-closer-scrutiny-11590598710) includes a picture of the new transformer being loaded on a truck to be shipped to SNL. The utility that ordered this transformer is missing a transformer which potentially affects grid reliability (the reason the transformer was ordered in the first place). Obviously, DOE would not have intercepted the transformer, which was a very expensive proposition, if DOE wasn’t concerned about the security and integrity of the system which provides power to critical portions of the US grid.

As the afore-mentioned government organization and I have surmised, the most likely concern is the Chinese transformer contains hardware backdoors to create a cyber physical event that damages, interrupts, or compromises the transformer operation. This type of exploit is significantly different from previous Chinese cyber incursions intended to steal data and intellectual property. Installing hardware backdoors inside power transformers have one purpose: to affect the transformer operation behind cyber monitoring and protection and even behind engineering safeguards protection. That is, the hardware backdoors are behind the OT Maginot Line which is why Executive Order 13920 only addressed hardware and control systems and explicitly excluded all network monitoring (https://www.controlglobal.com/blogs/unfettered/ics-cyber-security-is-the-second-coming-of-the-maginot-line-and-the-chinese-have-breached-it/). The importance of the industry knowing if there are hardware backdoors in the Chinese-made transformer sent to SNL cannot be minimized. If there are hardware backdoors in the Chinese-made transformer at SNL in addition to the known backdoor in the Chinese-made transformer installed at a US utility substation, the question then becomes how many other Chinese-made transformers already installed in the US grid (and elsewhere) have hardware backdoors? This is not trivial as the US has imported more than 200 large Chinese-made power transformers. These large power transformers are big, expensive pieces of equipment with long lead times that are not quickly replaceable. What can the utilities that own the Chinese-made transformers do considering they cannot replace them in any reasonable timeframe without impacting grid reliability? What about utilities with knock-off transformer equipment from China that may also have backdoors and not work as expected (https://www.controlglobal.com/blogs/unfettered/the-chinese-hardware-backdoors-can-cause-transformer-failures-through-the-load-tap-changers )? In https://www.controlglobal.com/blogs/unfettered/information-sharing-on-control-system-cyber-incidents-is-not-working-and-that-can-be-deadly /,a case was provided where the Chinese had installed a shadow control system sensor network in a pharmaceutical facility. It was unclear if the purpose of the shadow network was just to steal data or also to compromise the drug-making operation. However, it begs the question what other Chinese-made equipment in critical infrastructure applications have hardware backdoors or other unknown connections?

Generally, when I give presentations, I include examples of actual control system cyber incidents. Often, I will get responses from attendees who tell me they have experienced similar incidents. My presentation to the January 26, 2021 Public Safety Canada ICS Cyber Security Conference was no exception. Following the presentation, I was told of a case where a process sensor problem led directly to a combustion turbine blowing up. These types of issues generally are not understood as being cyber-related and are often not shared within their own organization much less with the rest of industry. This lack of internal information sharing explains why my database is not public.

Sharing information on control system incidents was not a problem until cyber came along. When I was at EPRI in the late 1980’s until 2000 (we started the control system cyber security program in 2000), EPRI personnel (including myself) held or attended many control systems and maintenance conferences where lessons learned from control system incidents were integral parts of the conferences. Unfortunately, that openness on sharing control system incidents changed when control system cyber security started being considered a national security issue. This reticence to share brings us to today where control system cyber incidents are not identified or shared.

What does it take to get people to be educated and share control system cyber incident information in a non-attributed manner? It can, and has, been done. The reliability and safety of our critical infrastructures depend on the identifying and sharing of control system cyber incidents.

Joe Weiss