MES: When Preparation Meets Opportunity

May 14, 2009
An Innovative Implementation of MES Is How Bristol-Myers Squibb Ensured That Activities Performed at FAT Became Useful in Meeting Regulatory Requirements

By Brian Chviruk, Bristol-Myers Squibb

The Factory Acceptance Test (FAT) represents a lingering dilemma for many engineers: Accept the vendor’s standard FAT with the intent to recheck everything during the Site Acceptance Test (SAT); develop specialized FAT procedures intended to help reduce SAT activities; or choose one of any number of other options.

When Bristol-Myers Squibb (BMS) announced plans to construct a biologics manufacturing facility in Devens, Mass.,  it provided a unique opportunity for BMS engineers to put into practice some intriguing ideas they had been discussing about streamlining the entire validation process by incorporating FAT activities. A central enabler included in these discussions was the early incorporation of a manufacturing execution system (MES) into the work process.

Validation Defined

“Validation” has many different meanings depending on the industry sector. For example, many people associate validation activities with a strict set of requirements established by the U.S. Food and Drug Administration (FDA) that apply only to the drug and health-care industry. Though those in particular industries are quite familiar with FDA validation requirements and activities, the fact is that various forms of validation exist in all industries, albeit frequently referred to by other names – “commissioning,” “verification” and “ISO 9000-certified” being among the more familiar.

Validation is closely related to quality-release testing, but is different in the sense that the latter often implies post-production verification that the product meets pre-defined specification, whereas prospective validation activities prove that the process itself produces products within specification.

Validation also dictates that a robust quality assurance plan exists, is being appropriately applied and, therefore, proves (validates) that the process will produce products within pre-defined specifications.

The significant requirements of any quality assurance (validation) effort include:

  • A set of procedures that cover key business processes;
  • Monitoring processes that ensure procedural effectiveness;
  • Maintaining accurate, detailed records;
  • Checking process outputs and applying appropriate corrective and preventative action where necessary;
  • Regularly reviewing individual processes and the effectiveness of the overall quality assurance plan; and
  • Facilitating continual improvement.
Sustainability 2010 Goals in Brief

Bristol-Myers Squibb and the entire pharmaceutical industry face many challenges in meeting the needs of patients, health care providers and society at large. The entire industry must intensify its efforts to improve access to medicines and healthcare in general. It needs to encourage innovation, in part by protecting intellectual property. It must operate efficiently and profitably so it can continue to invest in research and development leading to new and better medicines, and it must do all of these things while advancing safety practices and environmental protection.

The Bristol-Myers Squibb Web site provides significant details about its Sustainability 2010 Goals including its progress in addressing key areas.

Once a process is running and making quality product, engineers are often reluctant to introduce new ideas, technologies or practices because of misconceptions about regulatory hurdles and/or perceived inconsistencies between systems or sites. What is odd is that those same persons often take pride in constantly refreshing their knowledge about technological advances while remaining guarded about being an early adopter.

While a cautious approach intended to protect consumers and patients is critical, today’s worldwide business environment also punishes those that procrastinate in their adoption of innovative technologies and business practices.

What we often witness is that 21st  -century business successes come to those companies with a corporate culture that embraces innovative thinking. However, innovative ideas alone aren’t the whole answer. Each idea must be submitted to the securitization of reliable, risk-based management practices that can transition innovative ideas into well-organized, actionable opportunities.

21st-Century Thinking

Shortly after the FDA began encouraging the use of risk-based approaches to advance good manufacturing practices, BMS engineers began exploring ways to increase the use of state-of-the-art technologies and practices. Although several innovations have been successfully completed, many of the more novel and forward thinking ideas simply weren’t feasible to implement on existing processes.

When BMS announced plans to build a large-Scale cell culture (LSCC) facility in Devens, Mass., a once-in-an-engineer’s-career opportunity emerged. Employees would have the opportunity to implement an often-discussed, highly integrated, paperless plant designed to achieve:

  • Highly automated and tightly integrated electronic production and laboratory batch records;
  • Batch releases by exception;
  • Application of a science and risk-based approach to validation;
  • Implementation of an open-office environment designed to maximize collaboration within and across functional groups;
  • Contributions to the company’s Sustainability 2010 Goals.

As previously mentioned, the key to turning innovative thinking into actionable opportunities is the use of sound, risk-based management practices. In the case of BMS’s LSCC facility, risk management considerations included using existing industry consensus standards such as ASTM E2500, ISA88 and ISA95.

Using Consensus Standards

The National Technology Transfer and Advancement Act of 1995 requires federal agencies (i.e., EPA, FDA, OSHA, etc.) to recognize existing consensus standards where available and applicable. That means that all government agencies have been instructed to accept the premise of consensus standards and abide by their requirements.

ASTM E2500 describes a science and risk-based evaluation process that is designed to improve how equipment capabilities and performance are validated. The standard achieves its objective by using quality management principles throughout the system life cycle. This approach required incorporating information developed by business owners during the conceptual phase, by the engineering and the equipment supplier during the design phase, and continues through start-up and operation.

ISA88 describes terminology and models that result in good design and operational practices of process facilities. The premise of ISA88 is to create a hierarchy of module classes that carry out a variety of equipment-centric functions. Depending on process requirements, module classes are orchestrated through a hierarchy of procedures and product recipes.

ISA95 describes the terminology and models for the integration between business (i.e., enterprise resources planning [ERP] and manufacturing execution systems [MES]) and process control systems. The standard provides details about structuring information using a standardized exchange format that is recognized by most ERP and MES applications. The ISA95 standard has proven useful for a variety of purposes including serving as a guide for the definition of user requirements, for the selection of MES suppliers and as a basis for the development of MES systems and databases.

Originally ASTM E2500 was developed by the healthcare industry and ISA88 for batch processes. Since then, both have proven applicable to other industry sectors, including continuous processes. Further proof of ISA88’s ability to benefit other industry sectors is demonstrated by its adoption by the Organization for Machine Automation and Control and the subsequent creation of S88.05 for use by the packaging industry.


Traditionally, the approach to getting instrumentation and control components from shipping box to on-line production has been referred to as “commissioning,” and it consists of a string of sequentially executed tasks that frequently begin with on-site activities, thus all but ignoring potential FAT contributions to the process.

Following careful examination and in-depth discussions, BMS engineers were convinced that the combined use of ASTM E2500 and ISA88 provided minimal risk and significant benefits to validating such a highly automated system.

In order for a risk-based process to work, all stakeholders―engineering, quality, maintenance and manufacturing―need to be able to roll up their sleeves and work collaboratively from day one.

Certainly the use of electronic tools facilitates efficient collaboration, but significant efficiency gains are also available through the use of pleasant communal “open office” spaces.

One of Bristol-Myers Squibb’s strategic goals is to advance its “environmental greenness,” and the Deven facility represents a significant commitment toward that goal. Constructed of energy- efficient materials, glass walls and highly efficient heating and electrical systems, Deven’s communal office spaces are flooded with natural light thus creating a pleasant yet efficient working environment.

Long before moving into the new Deven building, project team members initiated their own communal working environment by locating various departmental representatives in close proximity to one another. Amid the benefits of the communal project team environment was a recognition that eventually became one of the projects guiding principles: “Get the right data, at the right time, reviewed by the right person, once!”

Achieving that goal required that the project team wholeheartedly to commit to using and integrating the consensus standards ASTM E2500 and ISA-88. Their initial belief, which they later confirmed, was that the combined use of these two standards would result in an efficient streamlined approach that:

  • Made effective use of available resources;
  • Ensured early stage quality acceptance;
  • Integrated equipment and control system testing;
  • Used modular software design;
  • Reduced life-cycle costs.

The team also became convinced that the lynchpin to success relied on their aggressive and innovative use of a robust MES. (See A Streamlined Approach graphic below) Thus the decision was made to invest the up-front resources necessary to implement a site-wide MES solution and do it before any equipment or control system entities became ready for FAT.

Implementing Innovation

When you look at the product data sheets for any of the available MES solutions on the market, you won’t find a single one that includes a module designed to address verification testing activities. However, BMS engineers recognized that MES-based batch records and verification testing protocols are remarkably similar in that both include a set of instructions, space to record results and sign-off sheets.

Using the models defined in ISA88, Bristol-Myers Squibb wrote its FAT protocols in a modular manner using the Syncade Smart Operations Management Suite master recipe authoring tools from Emerson Process Management. Not only did this prove to be an efficient use of an MES in order to facilitate verification testing, but it also produced protocols that mirror Bristol-Myers Squibb’s production recipe architecture, thereby providing end users with an opportunity to become familiar with the Syncade system.

Each work instruction was designed to support the “plant in a box” concept and each resides in a library, thus facilitating reuse among systems. When a new protocol was required, it was assembled by dragging and dropping appropriate work instructions from the library and arranging them in the appropriate sequence. This approach resulted in higher consistency across documents and fewer preparation man-hours compared to the anticipated workload using a paper-based solution.

Every FAT protocol (unit procedure) began with protocol boilerplate―objective, scope and preliminary checks. Then common pre-shipment checks, such as general arrangement and piping and instrumentation diagrams and walk-downs. Other installation checks to ensure proper installation on-site were included (i.e., proper materials of construction, accuracy of dimensional and connection points, component accessibility, spare parts, etc.) The last portions of the BMS electronic FAT protocols included operational checks (i.e., power up, loop check, dry and wet checks, etc.) of system functionality.

Certainly FAT protocols vary between entity classes (i.e., Bioreactors, Centrifuges, CIP skids, Chromatography Skids), but the use of an electronic method for each entity class permitted it to be configured once and then re-used by redefining entity specific parameters at the instance of each test.

For the Devens project, approximately 50% of the electronic FAT protocols became “standard instructions,” meaning they were written to be used over and over. Another 20% were “variable instructions,” meaning each subsequent use required only minimal changes. The remaining 30% were “unique instructions,” required authoring and review times equivalent with a paper-based system.

Based on the time required to prepare paper-based documents for previous projects, and excluding the time required to learn the Syncade system, the use of the Syncade’s Electronic Work Instructions permitted the BMS project team to save more than two hundred man-hours. (See “Bioreactor Document Preparation Efficiencies” table below.)

Though GDP is important for a variety of regulatory reasons (i.e., EPA, OSHA, legal, etc.) in every industry segment, the BMS project team focused its GDP efforts on complying with FDA regulations and guidelines. Nevertheless, it should be understood that proper GDP is proper GDP, regardless of the industry.

GDP is not specifically addressed by the FDA. Instead, GDP repeatedly appears in cGMP (current good manufacturing practice) documentation and can be paraphrased as, “Good documentation practices are the set of activities that enable you to record product-related data, hand-written and/or electronic, in a legible, traceable and reproducible manner.”

When using paper-based documentation, GDP is often one of the most common and time-consuming source of errors. These errors include comment and/or signature blocks that are incomplete or illegible; failure to follow “cross-out” procedures intended to allow others to read what was crossed out; and documents that become illegible because of tears and smudges. And then, of course, there is always the possibility that the paper record is misfiled or becomes completely lost.

For BMS the use of the Syncade and its typical electronic insistence on properly completing forms played a significant role in eliminating the many disadvantages associated with paper-based documentation systems. Additionally, Bristol-Myers Squibb’s MES-based solution was able to make use of pull-down menus for such things as standardized “as found” responses and pre-defined comments. And lastly, the Syncade application was hosted on a secure and routinely backed-up server, thereby eliminating those dreaded misfiled and lost records.

Everyone knows that the job isn’t “done-done” until the paperwork is finished. In the case of FDA regulated products, that means a product can’t be released for shipment until authorized persons have reviewed the production data, operator comments, laboratory results, etc., and reconciled each and every exception. When batch records are paper-based, it means going through every page, deciphering every signature and comment, probably calling a few people for clarification of what they meant, and so forth―a tedious and often time-consuming string of activities.

Contrast that with documentation that is being completed and posted on a server in real time. This documentation is consistent in appearance and content from shift to shift and operator to operator. Also, consider the benefits of having the MES assemble laboratory and manufacturing records in consistently organized batch records with exceptions clearly indicated. The result is that the MES wholly facilitates Bristol-Myers Squibb’s “batch release by exception” goal.

LEED Certification

When BMS committed to constructing its Devens, Mass., LCSS biologics manufacturing facility, it also committed to the pursuit of the U.S. Green Building Council’s Leadership in Energy and Environmental Design (LEED) certification, which recognizes site development, water savings, energy efficiency, materials selection and indoor environmental quality.

BMS engineers wholeheartedly embraced the company’s commitments to implement long-discussed plans to create a highly integrated paperless plant. The results include not only electronic production and laboratory batch records supporting exception-based batch releases, but also a streamlined risk-based validation process, all the while supporting the companies Sustainability 2010 Goals through LEED certification at the Deven facility.

In short, success truly is what happens when diligent preparation and excellent execution are applied to an opportunity.

Brian Chviruk is a Validation Engineer at the Bristol-Myers Squibb, Devens, Massachusetts, Large Scale Cell Culture facility

Bristol-Myers Squibb and the entire pharmaceutical industry face many challenges in meeting the needs of patients, health care providers and society at large. The entire industry must intensify its efforts to improve access to medicines and healthcare in general. It needs to encourage innovation, in part by protecting intellectual property. It must operate efficiently and profitably so it can continue to invest in research and development leading to new and better medicines, and it must do all of these things while advancing safety practices and environmental protection.
The Bristol-Myers Squibb Web site ( provides significant details about its Sustainability 2010 Goals including its progress in addressing key areas.