FAT and Furious: Cargill Wastes No Time

By Jim Montague

Oct 03, 2018

CT18 EmrEx Header

It might seem like a factory acceptance test (FAT) is a heck of a time to think about other opportunities, but a really effective FAT can pave the way for gains on multiple fronts. If it helps, just think of it as extra credit. For example, Cargill reports it recently carried out FATs at two plants with help from Emerson Automation Solutions, and the similarities and differences in those projects enabled the two longtime partners to devise five major lessons for conducting the ideal FAT and achieving related gains.

The two firms have had a strategic relationship for 10 years as Emerson has helped Cargill migrate six plants in different stages from its legacy Provox and RS3 controls to a modern DeltaV DCS. Their agreement covers control system, electrical and instrumentation services; cabinet fabrication; machinery protection with AMS 6500; safety instrumentation system (SIS) for burner and high-integrity interlock applications; and Mimic simulation software.

"This is really a tale of two FATs—one at Cargill's steephouse plant in Cedar Rapids, Iowa, and the other at its oil extraction plant in Memphis Tenn.," said Katherine Niehoff, process automation systems engineer, Process Systems and Solutions, Emerson. "Typical accessories for both projects included checkout from templates, Mimic simulation, and that they were operator driven and relied on our long-term partnership."

Niehoff and Dan McDonald, automation programmer, Cargill Corn Milling, presented "Don't Waste My Time—Effective FAT Strategies" this week at the Emerson Global Users Exchange in San Antonio.

FATs domino effects diverge

The Memphis project was a new oil extraction application in an older plant, which relied on testing performed by three dedicated subject matter expert (SME) operators during a FAT for a similar application in Omaha, Neb. This earlier FAT was completed reasonably on schedule, and the resulting system was made available for operator testing in Memphis.

Meanwhile, the Cedar Rapids project was carried out in a windowless, cinderblock, supply room, located adjacent to the plant's control room, where testing was done by one lead SME operator and one to three others rotating in and out. This project included remote support from Emerson, and "tenacious testing the dragged on and on," Niehoff said.

"In the end, the Cedar Rapids steephouse was a breeze, and got done three days early, while the Memphis project went on for an extra three months," said Niehoff. "The main lesson we learned was that time well spent early prevents extra time spent later."

McDonald reported the delay in Memphis was due mainly to differences in programming languages and terminology used. "When you're writing code, saying 'not closed' doesn't mean the same as 'open,' and resolving these differences takes time," he explained.

Five FAT takeaways

McDonald reported that Cargill and Emerson distilled what they learned from these two recent projects into the five most useful strategies that should not be skimped on for a successful FAT:

  • Dedicated time/location for the FAT, which means performing it far enough away from the main control room and plant site. This is preferable because it allows the testers to focus on the FAT, and not get distracted or drawn into routine plant operations. That said, the testers may also want to set up a "phone a friend" capability, so they contact the plant and operators or visit periodically to verify testing scenarios. Testers and operators must be supported in these efforts.
  • The right attendees in a successful FAT include: process engineers who can approve proposed FAT updates and already know the design documentation and review process; automation support people who can get 2 a.m. troubleshooting calls, but need training on new programming just like the operators; standards representative who can make sure processes are executed consistent with company practices; and managers who can compare the cost of testing to the value received. However, the most important FAT resource is operators, who know the plant and its problems best, can provide crucial buy-in, and provide the input that earn the most support from operations.
  • Modern formats including electronic logs and check out forms, which may be able use non-traditional scheduling either pre- or post-FAT. These tools include Microsoft Excel spreadsheet software and Access database software, of course, and SharePoint collaborative software, as well as Microsoft Office 365 and other cloud-sharing programs.
  • Realistic, procedural and "break it" testing of every configured detail and scenario. This includes testing from the ground up of all graphics, databases, complex loops and equipment module (EM) steps for running programs, as well as trying break these functions. Operator tribal knowledge of "blue moon" events and responses can be a big help at this stage. Likewise, simulation software like Mimic support the FAT by allowing testing of real-world scenarios for the whole plant or specific areas.
  • Other improvement opportunities, including process optimization, alarm management and trying new technologies, such as Alarm Mosaic big picture-view software; Dynamic Alarm Module software to eliminate individual alarms on heavily instrumented equipment; mobile worker devices, including the DeltaV mobile app for an intuitive, native mobile user experience on those devices.

"FAT is all about doing things the right way, but it can also be a time to look at doing things better, whether it's meeting the goals of the daily grind or transforming processes on a grander scale," added Niehoff. "This can also be a period to learn about and try new technologies by just taking one more step.”