Distributed Control / PLCs & PACs / Systems Integration / Optimization

Control System Evolution at Bayer: 24 Years and Counting

Bayer Healthcare Partnered with ABB to Bring its 1980s-Era Control Systems into the Future

By Walt Boyes

ABB A&PW 2012

The Bayer Healthcare plant in Berkeley, Calif., was built in 1903 and has produced everything from black-leg and smallpox vaccines to its current product, Kogenate, a blood-clotting agent for persons with hemophilia. 

"In more than 24 years of partnership with ABB, Bayer has managed a program of continuous upgrades to its control systems," said David Kavanaugh, process control systems engineer for Bayer, who, along with his colleague Mike Kyllo, discussed the company's migration journey this week at ABB Automation & Power World 2012 in Houston.

"Efficiency and flexibility are now strategic requirements for plant automation," Kavanaugh said. "Originally, our systems had a low level of automation with no closed-loop or advanced control and with poor standardization. Automated sequences were hard-coded and inflexible. Code was poorly documented, if at all. Interconnection between different manufacturers' systems was difficult, and it was hard—or flatly impossible—to automatically get data from a system."

"Today's plant," Kavanaugh said, "has on-line and at-line process analyzers and advanced control has been implemented. The use of OPC and standard third-party drivers has made interconnection and data transfer straightforward. Electronic batch records with recipes are selected through manufacturing execution systems. We have self-documenting tools and auditing, and our systems are integrated in a site-wide network among different plants and utilities and buildings, and with IT. Data can be automatically transmitted and web services are implemented."

"How did we get here?" Kavanaugh asked. "There were three basic paths open to us. We could maintain our older MOD and Process Portal systems. If we did that, we would have no capital expenditures, no additional training and no problems with operator familiarity. But we would also have higher maintenance costs, an inability to meet new strategic functionalities, and we'd face ever scarcer and ever higher- priced spare parts.

"Or we could gut the existing systems and do a ‘rip and replace.' That would give us the functionality we needed, ensure longevity of systems and software, and the lower maintenance costs associated with new systems. But we would also have production downtime, large capital costs and increased training requirements."

Kavanaugh said that Bayer chose the third path. "We chose to evolve," he said. "This gave us new features, more flexibility and less time pressure. We think it was a good trade."

What does evolution look like? Changes are occurring through concurrent multiple projects: MES, DCS upgrades, batch implementation and others, as Bayer staff learn the new skills required for new technologies.  Bayer is adding new functionalities to its systems (connections to other systems, remote desktop, advanced control, etc.), making hardware and software changes standardized across systems, and learning to partner with internal and external suppliers.

"We are producing a new, site-wide automation platform that will integrate all the existing DCS systems," Kavanaugh said. "Upgrading to the latest 800xA version will allow that integration. We're integrating our systems to MES for electronic batch records, based on the ISA95 standard. We will add new, small, scalable controllers (ABB's 800M) for on-skid installation while using existing controllers and logic for the remainder of the plant."

A flexible integration connector will also allow future integration with other plant information and control systems. "We're implementing open-standard automation interfaces, including OPC and fieldbus, to be less dependent on a single vendor and to be ready for PAT integration," Kavanaugh said.

The introduction of ISA88 batch technology will prepare Berkeley to become a multiple-products site through flexible-control recipe management. Plus, the use of object-based development with reusable library objects from the Bayer Standard Libraries will allow streamlined implementation and validation, and will simplify system documentation.

A funny thing happened along the way, Kavanaugh noted. "We now use standard IT hardware and software [for our automation platform]," he said. "We need the same services as IT: anti-virus, operating system patches, application patches, performance monitoring and so on. We talk the same talk: TCP/IP, firewall, switches, routes and permissions."

Because of this commonality between the process control system and IT worlds, the process folks found they must follow IT's rules—and it hasn't always gone smoothly.  "They understand Windows," Kavanaugh said, "but they may not know process control and its special requirements."

"The IT department said they can provide on-call coverage," Kavanaugh explained. "But when we investigated the on-call procedure, we found that the best response time available was four hours.  Escalation procedures and support costs also were issues.

"We are working on training the IT department on control system technology and the engineering department on IT. We're supplementing IT support with engineering on-call support and have established service level agreements (SLAs) that specifically outline responsibilities and the escalation of issues. We've purchased warranties from vendors for onsite support, again with well-defined SLAs."

What's next for Bayer Berkeley? Kavanaugh said that there would be continued evolution of the control system platform for flexibility and connectivity. "We're investigating the virtualization of process control systems and a new site-wide historian. We're looking at new operator interfaces and technology like tablets, PDAs, smart phones and cellular. We want to look at remote connectivity and access through web-based applications. And we want an even tighter integration between the process control and IT groups."