Genentech Wireless Network Monitors Critical Assets

Aaron Hand is Managing Editor of Control Design and Industrial Networking. He joined Putman Media recently after almost 20 years covering high-tech industries, including semiconductor, photovoltaics and related manufacturing technologies. He has a B.A. in journalism from Indiana University, Bloomington, and an M.S. in journalism from the University of Illinois, Urbana-Champaign.

Check out ControlGlobal.com on Google+

As the U.S. R&D center of excellence, the campus in South San Francisco holds a special place for biotech company Genentech. The site—with about 10,000 employees working in more than 40 buildings spread across about 2 miles—also comes with special facility monitoring needs.

"The facility monitoring system is pretty unique, even within Genentech," said Vikas Bakshi, program manager for monitoring in the South San Francisco facility. At other Genentech facilities in California and Oregon, they use static systems that are monitored by security. "We have a dedicated facility monitoring system that lets us pull data and serve our customers."

Bakshi spoke this afternoon at the Yokogawa Users Group Conference in New Orleans, explaining the justification, design and implementation of the 1500-radio network used at the South San Francisco campus to monitor equipment in pharmaceutical research and pilot production. He described a scene before the changeover of an aging wired infrastructure, organically grown over the 30+ years of life of the diverse set of manufacturing, clinical, research, development and utility buildings.

Mobile Equipment Difficult to Track

Depending on business needs, mobile equipment moves throughout the campus. "A lot of our freezers would move from one building to another," Bakshi said. "How do you track the identity of that equipment?" It involved a lot of paperwork, and there was constantly discontinuity.

The moving equipment also caused other issues, Bakshi added, including damaging the infrastructure as brittle wires were pulled and broken. He described wired networks in which it was difficult to identify where wires led to and from.

"About five or six years ago, we started talking about implementing a wireless solution," Bakshi said. "It would help us move all of our equipment with the least amount of impact to the data and the users. And it would increase our monitoring capabilities."

"We partnered with Yokogawa, which came in and did a proof-of-concept to show that yes, you could have a freezer or incubator on a radio." Genentech's Vikas Bakshi on the company's successful implementation of a wireless monitoring network at its South San Francisco campus.Genentech launched a multi-year initiative to do exactly that, and it's been about four years in the making. The team assessed four suppliers and ultimately settled on Yokogawa. At the time, none of the companies had an existing solution that fit what Genentech wanted to do; only products they were thinking about developing.

"But we partnered with Yokogawa, which came in and did a proof-of-concept to show that yes, you could have a freezer or incubator on a radio," Bakshi said. The team got funding of about $4 million for the project. And because it was essentially an untried technology, they focused first on R&D areas where non-validated equipment was used. They did not want to risk FDA violations where good manufacturing practice (GMP) requirements were in place.

"We defined our scope to be very limited," Bakshi said, outlining a target capability of 2500 radios with up to four channels each, building a campus-wide infrastructure for the non-GMP labs.

Immediate Benefits Realized

The project faced several constraints, including a radio frequency limit of 900 MHz set by IT, mobile capability for equipment requiring minimal reconfiguration changes, reduced cost of per-installed channel, a manageable number of infrastructure units, and battery backup capability so that, during the move, they could continue to receive data and monitor the units.

The 900-MHz I/O radios monitor temperature, compressor amps and door status on ultra-low temperature freezers and incubators throughout the campus. "The benefits for us were pretty immediate," Bakshi said, referencing the brittle wiring that often broke during moves. "The radios have defined codes that let us know if troubleshooting is required. We can tell where the breaks are."

The radios use a self-healing and self-organizing mesh network to allow equipment to be moved seamlessly between buildings without the need for any reconfiguration. "Each equipment move resulted in significant data administration to reconfigure the monitoring system to track the alarm and trend data," Bakshi said, describing the old system. "Now radio assignments require only minor changes in the location data for the equipment as moves occur. The alarming and trending capabilities do not require configuration, saving significant data admin costs."

Although the initial investment was "pretty scary," Bakshi conceded, the $2500 up-front cost for a radio is nothing when considering the hundreds of thousands of dollars worth of samples those radios are protecting. Further, Genentech is expecting to save about $25,000 a year in direct costs with the system in place.

GMP Operations Are Up Next

For facilities considering a similar move, Bakshi warned that it's a project requiring dedicated resources. "You have to define an engineer within the organization to get to the delivery you need."

He also said that the project required an open mind to what the design team was saying. "We knew what data we wanted to get in, but we had very few ideas about how to get the data. We left it up to the design team."

For the future, Genentech wants to expand the network not only through more of the non-GMP equipment, but it wants to implement a validated wireless network as well. "We want to implement it in the GMP world," Bakshi said. "We hope to be able to implement that next year."