Cybersecurity / Industrial Ethernet

Why cybersecurity cannot be overlooked

Effective cybersecurity requires ceaseless monitoring and evaluation of network data and traffic to identify and head off evolving intrusions and potential attacks, but improved tools are making it simpler and easier.

By Jim Montague, executive editor

Security isn't a barrier. Security is awareness.

But it's hard to get out of that old barrier mindset. After all, people have been hiding in caves, huts and castles, and behind shields, fences, walls and locked doors for a long time. The desire is always to get behind the barrier where you can breathe easy.

However, that's not how cybersecurity works. Today's probes, intrusions and attacks are directed by software, travel via increasingly widespread Ethernet networks aided by Internet protocol (IP), and can run on all kinds of computers that are always up and running—so they must be continuously sought and eradicated in the same way.

"You can put firewalls between the plant floor, IT/business level and other functional areas that define the traffic allowed between them, but then you have to monitor that only the right data gets through in the right way, and that means using the firewall's whitelisting functions to define specifically what data sources and destinations are allowed," says Chuck Tommey, PE, business development manager at A&E Engineering, a CSIA-member system integrator in Greer, S.C. "If there's unusual traffic, whitelisting can monitor it, not just at the firewall level, but down to the PC level, too. Then, IT can call application control, take an inventory of all the files and executables on the PC as it's running, quickly give signatures only to those allow to run, and not allow them to be modified.

"Unlike typical antivirus software that's reactive, and blacklisting that just blocks known bad actors and must be constantly updating, whitelisting doesn't need to repeatedly download new versions, apply patches, and conduct virus scans. When whitelisting software is updated, new signatures are needed for applicable files, and this does require some effort, but IT has been using these tools for a long time, and the plant can use them, too. In fact, today's plants look a lot like IT departments with all their servers, credentialing and software files, but this means they can also use newer tools like security information and event management (SIEM), which sorts monitoring log files from firewalls, switches with routing functions and PCs, looks for any unauthorized activity, and alerts the administrators."

Neil Peterson, DeltaV product marketing director at Emerson Process Management, adds, "There's no silver bullet for cybersecurity. Everyone want's a secure solution, and it's possible to make process control systems stronger, more secure and greatly reduce risks, but there are no guarantees. So, cybersecurity assessments, risk prioritization, firewalls and antivirus software, network segmentation, whitelisting and constant monitoring are crucial. Tools like SIEM can show all networking monitoring in a central display, and look for anomalies in how the network is supposed to be working. Users can then write rules for good or bad behaviors in their networks. This is very doable for steady-state process applications."

Vulnerabilities and vectors

Today's stressful cybersecurity environment evolved from IT-based computing and networking, which historically sent and received data from all devices on their networks, and tried to put a lid on afterwards. At first glance, the new and expanding Internet of Things (IoT) and Industrial IoT (IIoT) just mean more pervasive networking, which means "larger attack surfaces" and "more attack vectors"—in short, larger targets and more places for bad guys to shoot from. The predictable result has been a stream of mainstream and a few industrial breaches, most recently at Target department stores, at least two U.S. government agencies, an unnamed German steelmaker last year, a demonstrated hack of Jeep's on-board automotive systems, and who knows how many undisclosed events.

Germany's Federal Office for Information Security (BSI) reported in December, 2014, that staff at a steel mill were subjected to a targeted-email, or spear phishing, attack that reportedly tricked recipients into opening an attachment or visiting a website that downloaded malware. This code then penetrated and compromised many of the mill's corporate and production systems, and even prevented the shutdown of a boiler and blast furnace, causing severe damage. This was the first well-known disclosure of malware causing physical, plant-floor damage since 2010, when Stuxnet was found to have damaged uranium centrifuges in Iran. (A report by the SANS Institute on the attack can be accessed here.)

"The German steel mill incident and the stream of vulnerabilities discovered in the OpenSSL library, including Heartbleed and others that received less public attention, or the Poodle vulnerability in certain common configurations of SSL/TLS stacks, have had a significant impact on the industrial control systems (ICS) community," says Ragnar Schierholz, head of cybersecurity for ABB's process automation division. "All these events and discoveries remind us that we can't rest on achievements we've made, but must stay constantly alert, and proactively look for possibilities to improve the cybersecurity posture of our ICS products and installations.

"The trend of increased attention to the ICS domain in the hacker communities continues. Flagship hacker conferences such as BlackHat or DefCon rarely go without talks and demos where security researchers describe new approaches to hacking ICSs. These aren't just using newly discovered vulnerabilities, but are rather inventing new attack strategies and approaches. For example, work by Alexander Bolshev and Alexander Malinovsky looks at first like yet another hacker manipulating industrial field-level protocols—in this case HART. However, at a second look, one realizes it's actually a radically new attack strategy. Instead of the usual top-down approach from the Internet into the target organization's office to the industrial control system, Bolshev and Malinovsky go bottom-up, and attach to the field-level networks, compromise workstations connected to the field devices, and work their way up to the ICS or even the office environment and enterprise. This circumvents the most common layers of defense-in-depth architectures, which assume that attacks come top-down, and assume that the inner core of the system is trustworthy."

Jeff Melrose, principal technology strategist for cybersecurity at Yokogawa Corp. of America, adds, "All automation users should follow the ISA99 standard recommended zoning of their industrial networks. They should have a very robust, Level-3.5 DMZ separation area between business networks and plant networks. This separation between plant and business networks is critical to dealing with sophisticated attacks, and would have thwarted the German steel mill breach from causing physical damage. A double-firewall-mediated DMZ is the absolute minimum for plant network security separation, but users should also consider more sophisticated methods such as differential media gateways, application proxies, advanced VPN provisioning and unidirectional gateways."

The brightening side

The good news is that improving strategies, protections, and even IIoT, virtualized computing and cloud-based services can also make cybersecurity easier. Just as eternal vigilance is the price of freedom, effective cybersecurity is always on, always monitoring, and always examining data and network performance for unauthorized and unusual activity—the traces and footprints of intrusions and potential attacks.

Consequently, updating passwords, closing unneeded USB ports, segmenting networks with managed Ethernet switches/firewalls, encrypting data traffic, adopting a software patching policy and training staff are all essential security practices, but follow-ups must be identified and performed because the threats never stop adapting. As viruses, worms and denial-of-service incidents are joined by spear phishing, man-in-the-middle, watering-hole, access-and-invite, and reassembled-malware attacks, the remedies need to stay ahead.

Luckily, innovation and improvements also work on the side of the angels. So, just as the IT side has developed more effective security tools and policies, users in process applications can follow their lead. At the same time, there are many cybersecurity solutions making many sometimes empty claims, so individualized security risk assessments and evaluations of solutions are also crucial.

"I encourage practices that defeat the fundamental way Stuxnet functioned, but industry has real threats that should be a larger focus," says Brian Foster, senior engineer at Concept Systems Inc., a CSIA-certified system integrator in Albany, Ore. "A recent example was a water-holing attack that targeted an ICS facility by compromising a nearby restaurant’s website, The compromised website was used to install Havex when the facility's staff ordered from it. Havex is a Trojan horse program that often targets ICS facilities by identifying OPC servers and attempting to exfiltrate collected data. It's often used in an attack's discovery stage.

"Many other threats exist, such as Black Energy 2 and Sandworm, but trying to protect against individual malware is no more effective than trying to keep your dogs out of the garden with a fence around one corner of the perimeter. Fortunately, the ICS community seems to be recognizing this need, and is turning towards robust, sound network architecture and strategies for protection against advanced persistent threats (APTs). Luckily, as threats continue to grow, useful defenses are becoming even more accessible to industrial users. For example, Tofino's managed Ethernet switches can be placed throughout an ICS as firewalls, and do deep packet inspection (DPI) for segmented protection; Cylance Protect's new antivirus software goes beyond black/white lists, and uses intelligent, neural network programming to analyze what programs are doing, determine if they're trying to modify files, and decide whether to lock them out or not; and Bedrock Automation just released an ICS that authenticates software commands all the way down to the chip level."

Recognizing this new reality, more users, integrators, suppliers and governments are demanding cybersecurity solutions. "We just learned that New York State's Environmental Protection Dept. has mandated that any water/wastewater plant with an IP connection must undergo a cybersecurity audit," says Dan Schaffer, business development manager for networking and security at Phoenix Contact. "This means a lot of applications will pass from using external firewalls/switches like our mGuard to protecting the next layer of the onion with more antivirus, anti-malware and whitelisting functions on the inside.

"For instance, Common Internet File System (CIFS) is a standard file-sharing method for computers that can establish a baseline for what a PC is supposed to look like, and then make sure that no critical changes are made to programs, executables or HMI files. If you're running antivirus software, then CIFS can let it scan plant services and PCs through an mGuard switch."

Beyond basic services

Once passwords are updated, ports are closed, the network is segmented and firewalled, traffic is encrypted, new tools are applied and staff is trained in basic security practices, what's next? Well, it's time to reassess, seek more vulnerabilities and protections, and resume seeking to make the odds of a successful intrusion and attack ever smaller.

For instance, one of Boliden's mining and smelting plants in Sweden already had a cybersecurity program for its System 800xA control system, but managers recently wanted to augment it with a fresh perspective, so they initiated ABB's multi-layered Cyber Security Fingerprint program to validate the plant's existing security policies, find new areas they hadn't considered, and supplement their efforts to prevent unauthorized access and mitigate computer viruses (Figure 1). The fingerprint service collected data from more than 100 points in the plant's control system, conducted in-depth interviews with Boliden personnel, and used software-based analysis to compare its performance with industry standards and best practices. It also helped the plant add protection layers to its 800xA system, and Boliden's staff reports it gave them greater confidence in their cybersecurity program and increased their knowledge about proactively preventing breaches.

"The Cyber Security Fingerprint should give enough of an overview to point out the biggest risks," adds Schierholz. "Such an overview should include the operational setup, such as policies and procedures, roles and responsibilities, as well as the technical solution as deployed, such as an inventory of network nodes, installed software versions, security updates, active user accounts, OS and application configurations, network configuration, etc.

"The documented as-is overview can then be compared to a recommended should-be status—commonly referred to as the configuration baseline, which should be available from the system integrator or ICS vendor. The resulting deviations should be assessed for impact in light of the specific application context and then countermeasures for the biggest risks should be designed and implemented. A more mature organization will already have an inventory, and should continuously monitor the operation in accordance to defined policies and procedures, such as timely deployment of qualified security updates, continuously control access including revocation of obsolete access permissions, and continuously monitor the threat landscape to adapt countermeasures to new, evolving threats."

Likewise, Emerson has developed a portfolio of cybersecurity services to help its Ovation control system users in the power generation industry secure their assets and meet compliance rules like NERC-CIP. These services include annual cybersecurity assessments, vulnerability identification and suggested remedies; scheduled deployment of patches and antivirus definitions, backup verification and preventive maintenance; assistance in creating, evaluating and implementing security and compliance programs; customized fleet service for single or multiple locations; and an incident response service for emergencies, which provides root-causes analyses, recovery and system hardening functions.

Similarly, Honeywell Process Solutions recently launched its Industrial Cyber Security Risk Manager, which is a digital dashboard designed to proactively monitor, measure and manage cybersecurity risks for process control systems within and across all security zones of a plant, including third-party systems. By understanding security zones, it's aligned with ISA 62443 and can calculate accurate risk scores. It translates cybersecurity indicators into measurements and key performance indicators (KPIs), while its intuitive workflow allows users to create customized, risk-notification alerts, and perform threat and vulnerability analyses, so they can focus on managing risks for reliable plant operations.

"It's still very useful to think of cybersecurity like process safety: ask what are the consequences if a system gets compromised; perform risk assessments with frequency and severity; decide which consequences are acceptable and which are not; and limit them to acceptable levels," adds Mike Baldi, cybersecurity solutions architect at Honeywell. "After setting up sufficient firewalls and monitoring, it's also important to have a site incident response plan (SIRP), including an isolation strategy for how to disconnect when a breach occurs, an approval plan for who can act when a incident happens, a team that can assess and do recovery, and a schedule for practicing the SIRP in the plant."

Josh Carlson, systems cybersecurity manager for North America at Schneider Electric, agrees, saying, "The same dots can be connected with cybersecurity as with process safety, but users also need to go beyond DMZs and segmenting their networks, and broaden their security scope to include outside engineers, contractors, vendors and components coming in. Stuxnet and its descendants are just messengers because, when they're plugged in and connect to the Internet, they pull down malware. This is going to happen a lot more in the future, and so staffers must be trained and given the tools to handle these threats."

To offer improved device security, Schneider Electric recently gained ISASecure Embedded Device Security Assurance (EDSA) certification for its Field Control Processor (FCP) 280 from the ISA Security Compliance Institute. FCP 280 is a distributed, optionally fault-tolerant, field-mounted process controller with alarming functions for Schneider’s Foxboro Evo process automation system.

Likewise, Tony Baker, security platform leader at Rockwell Automation, reports that it and Cisco recently added design guidance and validated functions to their Converged Plantwide Ethernet (CPwE) architectures to address constantly changing security practices. The new CPwE architectures leverage open industry standards, such as IEC 62443, and are accompanied by a whitepaper summarizing key design principles. They include "Industrial Demilitarized Zone Design and Implementation Guide," which provides guidance on securely sharing data from the plant floor through the enterprise, and "Identity Services Design and Implementation Guide," which introduces an approach to security policy enforcement that tightly controls access by anyone inside the plant, whether they’re trying to connect via wired or wireless access.

Better remedies for deeper defenses

Fortunately, stronger and simpler cybersecurity software and hardware tools appear to be multiplying lately, enabling users to effectively protect their applications and facilities more quickly and with fewer headaches.

For example, Bedrock Automation has developed a universal, open-source ICS with an electromagnetic backplane architecture, software-configured universal analog and discrete I/O, and embedded cybersecurity. Its redundant controllers run a SIL-certified RTOS, redundant smart power supplies and single/dual/triple I/O. All system modules interconnect on a patented electromagnetic backplane, which eliminates I/O pins, providing a new level of reliability, availability and performance. Its embedded Black Fabric Cybershield uses patented processor, memory, communications, interconnections, backplane and packaging to integrate security into every module.

"Bedrock's controller is easier to apply because it has built-in, galvanic isolation and isolation channels, so it doesn't need added relays. It can also can take care of its own encryption via OPC UA, too, because its backplane won't accept counterfeit modules," says Dee Brown, principal at Brown Engineers LLC in Little Rock, Ark., an electrical, mechanical and system engineering firm that focuses on water, wastewater, power distribution, critical power and security projects. "We're looking to add it to three or four utility customers. They want to control security functions from their iPads and control centers because their networks are getting hammered every day by probes and attempted intrusions, and this controller will give them another layer of protection beyond their firewalls and VPNs. At the hardware level, it powers up and checks for validated hardware. At the operating systems level, it checks that valid programs are running. All of these have to be OK for the controller to operate. Regular PLCs don't do this."

One of the primary trends fueling development of these new cybersecurity solutions is the increasing convergence of IT and operations technology (OT), according to Bill Mueller, senior project engineer at Banks Integration Group, a life sciences-focused, CSIA-member system integrator in Vacaville, Calif. "For instance, Cisco and Rockwell Automation’s partnership has produced familiar switching gear with several functions typical of enterprise-level network gear. In the past, segmentation and network isolation were accomplished with two network cards, two switches and separate subnets. With the newer switch gear, we can get similar results with half the hardware. Also, configuration wizards built in to the switches provide easy setup of virtual local area networks (VLANs) and data traffic-shaping functions. Another benefit for our clients is that the new systems can be maintained by traditional control engineers directly, and don't require involvement by corporate IT. This lowers the barrier to design more efficient and secure network topologies."

Another useful tool, SecurePlant, has been cooperatively developed by Yokogawa, Cisco and Shell, which announced earlier this year that they're going to deploy their comprehensive security management solution at 50 Shell plants worldwide. SecurePlant consists of delivering OS patches and antivirus pattern files for control systems, and providing real-time and proactive monitoring, along with help-desk operations. Supplier-certified Windows security patches and virus signature files will be distributed from a SecureCenter to the SecureSite at each plant via Shell’s existing global network. The real-time and proactive monitoring capabilities will enable centralized management of plant security.

Training humans for a united front

While new innovations can help users stay ahead of cybersecurity threats, one of the most powerful security tools is already inside most process organizations, but remains unused by many: cooperation between plant/process and IT professionals, and training each in the needs and best security practices of the others. More recently, due to the risks posed by many outside contractors' vulnerable equipment, users are giving them same training and protections deployed internally.

"The two most potent ways to improve cybersecurity are layering network architecture defenses and training the humans," says Concept's Foster. "Phishing and spear phishing come through email, and watering hole attacks trick users into downloading malware, so it’s crucial to educate employees and third-party contractors about how to deal with them."

A&E's Tommey adds, "Last year, one of our clients got a virus on their control system, and when they checked their laptops, they found that some of the young guys in operations had changed the boot sequence on the PCs from only loading from the hard drive to also loading from a flash drive. They did it so they could load and play games, but this also introduced the virus. These abilities can be disabled, but this is also a management issue. People have to be told and trained in the rules, and then they have to be held accountable. They need to be made aware that cybersecurity threats are real and growing, avoid the normalcy bias that hackers don't usually get in and then assume they won't ever, and recognize that intrusion attempts on the IT side happen regularly.

"I'm starting to see more practicing of cybersecurity, but there's still not a big demand for it. Sadly, it will probably take another big, tragic event to get people's attention. People don't want to spend $50,000-$100,000 on cybersecurity, even though a big breach could cost 100 times as much as security system. Until the CEO, COOs and CTOs put cybersecurity in the budget, it won't happen."

Mariam Coladonato, product specialist for networking security at Phoenix Contact, adds, "This all starts with plant manager sitting down with IT, getting together on the same, and developing a cybersecurity. IT can help the plant do risk analyses and penetration tests to find, prioritize and address vulnerabilities."

Virtual, IP-aided security?

Finally, one the core principles of cybersecurity in process control is the ISA99 standard, now IEC 62443, which directs users to achieve defense-in-depth by further dividing their process control networks and sub-networks into zones with common functions, and link them with conduits guarded by managed Ethernet switches servings as firewalls (Figure 2). However, these directives can now be get an added boost from virtualized and cloud computing assisted by Internet-based networking, which is also how they've fueled the emergence of the IoT and IIoT.

"The simple and practical method we've employed to provide a secure transfer of system data is to use a DMZ. Data from the secure manufacturing networks is collected by a server on the manufacturing network, and passed onto a second server providing the data on the enterprise network," says Banks' Mueller. "Recently, we've been taking this architecture one step further to provide remote support for our clients. A connection is provided through the corporate enterprise domain to a machine on the DMZ, either VPN or an application server such as Citrix. From there, a remote engineer connects through the DMZ to a virtual development environment on the manufacturing network. These virtual development environments are deployed and destroyed on an as-needed basis to support the manufacturing facilities.

"We'll also begin seeing network security move from the domain of a specialized IT group to being an integral part of all aspects of system integration. I believe that soon it will no longer be a choice to secure the communication links. Security will be built into devices and applications. Similar to how all WiFi routers now default to strong encryption, hardwired networks will require some level of point-to-point authentication and encryption. This level of authentication of secure channels has already started integrating into applications directly via trusts, HTTPs and pre-shared keys."

Sven Burkard, industrial solutions manager at Belden, adds, "If you thought an air gap was enough, it isn't because all it takes is one mobile hotspot to bridge it, so that gap probably doesn't exist. Users often say, 'We have a firewall, so aren't we secure?' However, a contractor, integrator, OEM or the user can unknowingly connect an infected  PC, or defective device or network interface card (NIC), or an accidental network loop can cause a debilitating broadcast storm. We can’t just install a firewall at the edge of the network and forget about security. We must harden the plant floor as well with defense in depth. Cybersecurity isn't just about hackers trying to penetrate the network, but it's more about improving and maximizing network reliability and availability. Reduced downtime means improved productivity and safety."

Coladonato adds, "IoT and Industry 4.0 means everything is more on the network, which means more targets and attack surfaces, so it's just that much more important to take all the cybersecurity measures we talked about earlier."

Like this article? Sign up for the twice weekly Control Update newsletter and get articles like this delivered right to your inbox.



Free Subscriptions

Control Global Digital Edition

Access the entire print issue on-line and be notified each month via e-mail when your new issue is ready for you. Subscribe Today. E-Newsletters

Biweekly updates delivering feature articles, headlines with direct links to the top news stories that are critical to staying up to date on the industry — company news, product announcements, technical issues and more. Subscribe Today.