"Look at the elements of system management," said Jim LaBonty, director, global automation engineering, Pfizer, in his session at the Emerson Global Users Exchange this week in Denver. "They're at least as important to operations as they are to IT systems," LaBonty said. Those elements include:
- Security patching that is up to date and meets vendor requirements
- Backups of critical data are performed routinely
- Disaster recovery is possible at all times
- Limited node downtime due to hardware or software failure
- Proactive monitoring of node performance and infrastructure
- Up-to-date virus and malware protection
- Critical hot fixes that may impact system performance are installed
"Security patching is very important these days. Backups and disaster recovery are critical," LaBonty said. "Availability is very important—you want to get your downtime to short windows, and get the plant going."
Why Pfizer is virtualizing
"You're taking lots and lots of servers and bringing them down to one. Engineering time is reduced from hours to minutes." LaBonty said. Jim LaBonty, global engineering – automation, Pfizer.
LaBonty worked with Richard Dedzins, automation manager at the Sanford facility to deploy virtualization technology "for managing things outside the control system, such as backup recovery, patch management, system health monitoring of aging servers," said Bruce Greenwald, DeltaV platform business development manager, Emerson. "It's a way to consolidate and put them in a virtual environment so they can be managed and updated from a remote location."The Sanford facility had good reasons for virtualizing its DeltaV servers:
- Eliminate end-of-life equipment and upgrade "hand me down" servers
- No need to stock or have to locate old hard drives
- Centralized monitoring, access and control
- Energy savings
- Out-of-the-box redundancy and high availability
- Significant time savings for new server deployments
- Reduced installation footprint to reduce rack space/cabinets
- Template-based deployments make life easier
- Testing new solutions/ideas requires no additional hardware
- Virtual machine (VM) snapshot rollback, if needed, is easy
"You're taking lots and lots of servers and bringing them down to one. It gives significant time savings over traditional server configurations," LaBonty said. "Engineering time is reduced from hours to minutes." Pfizer's two largest plant systems are now virtualized, and it's virtualizing six more this fall.
Dell VRTX loves DeltaV Virtual Studio
"One of the concerns about virtualization it that it is complicated," said Greenwald. "Bare metal servers have no graphical user interface, so IT people do a lot of work in command prompt." Instead, the Sanford application uses DeltaV Virtual Studio software to present the virtualization procedure in formats easily understood by control engineers. "Its integrated DeltaV Virtualization Application is designed for automation folks, and it supports both online and offline systems—all DeltaV workstation types, non-DeltaV servers and workstations, as well as virtual control hardware for offline training and development systems. It has out-of-the-box templates for virtual machine creation, high availability and disaster recovery.
"Our solution starts with the Dell PowerEdge VRTX shared infrastructure platform with four blade servers, a shared area network, integrated switches, and redundant power. It's tower or rack-mountable, gives us 40 Virtual Machines in 5U (12 in.) of rack space, and supports failover clusters and integrated replication."
The VRTX high-availability features include automatic restart of the VM if there's a hot failure, automatic replication to secondary cluster, and automatic data replication every five minutes between clusters. "Half of my VMs can go down, and I have a copy of them that's less than five minutes old," Greenwald said.
A set of auto-run scripts make creating a VM "very easy," Greenwald said. "It asks you what sort of solution you want to set up, shows you which blade and switch it's on. I can do it all from the install screen."
Template libraries are available back to DeltaV v9. "You choose the type of station—operator, application—get dropdowns for network setup, click OK and it goes to work," Greenwald said. "It takes 20 or 30 minutes to spin up a server, after 5 minutes of active time. "We've productized virtualization. It's very easy to do."
Sanford Plant Takes the Plunge
At Sanford, one VRTX with four blades and maximized SAN storage (10 TB) is connected to six DeltaV secondary networks and plant LAN, OPC and PLC networks. Along with the above mentioned management functions, the server supports a VM for system health monitoring and seven GE UNICORN remote Windows 7 VMs to support manufacturing operations.
LaBonty lists these benefits of the Dell VRTX/Virtual Studio solution:
- Energy savings reduces overall site expenses
- High availability with no additional infrastructure requirements
- Lower cost of ownership than traditional server/workstation installations
- Significantly reduces engineering time for new deployments and testing potential solutions/upgrades
- In alignment with overall Pfizer corporate migration to virtual solutions
- No corporate IT support required for deployment or day-to-day operation (local site automation team has it covered)
- Centralized administrative control and proactive monitoring
The implementation was not without challenges. The VRTX weighs about 180 pounds, so "use at least two people to handle it," said LaBonty, "and reference the Dell manual before you try to rack-mount it. And if you want to actually remote in, you may want to turn that on."
Then there is a learning curve. "New technology is always different, with different terminology. It may take a few weeks or months to learn, but it's not years," LaBonty added.
On the other hand, now at Pfizer's Sanford facility, "Everything is up to date—patches, definitions—there's no need for housekeeping, because the patch management servers are virtualized. It only takes a few minutes to spin up a VM."
