Food and beverage automation is undergoing a series of transformations, from more ethical livestock treatment to eliminating artificial ingredients and processes to improving the hygiene of irrigation systems in agriculture. But perhaps the most impactful and paradigm-shifting development in this industry is the emergence of lab-grown meat, which is changing not only how we view meat production and consumption, but also how we view the nature of food automation.
Understanding the Cultivated Meat Movement
Cultivated meat, also called lab-grown or cultured meat, grows mammalian and fish cells for human consumption in a process similar to biotech drug substance manufacturing. Cultured meat is real meat that is grown from a cell bank of a small number of real animal cells. It doesn’t require animals to be raised, fed, and slaughtered as traditional meat does.
In theory, cultured meat should be more sustainable than traditional animal-slaughtered meat. Bio-cultured meat requires significantly less land and water, has a better yield per pound of feedstock, and produces pure meat protein and/or fat that creates minimal waste and byproducts. A single batch can grow 350 pounds of pure meat protein in less than two weeks with no feathers, beaks, claws, blood, bones, or excrement to dispose of.
Traditional meat raises the animals where few people live and must be transported to a slaughter facility, processed and further transported to market. Cultured meat plants can be built much closer to population centers, reducing travel time and expense. Additionally, because it is grown in a sterile environment, it is cleaner and has a shelf life that is significantly longer than slaughtered meat – as most contaminants that spoil meat are introduced during slaughter processing.
Cultured Meat Automation Intersects with Biotech Automation
Growing cultivated meat is a relatively new concept in the biotech marketplace. Although this new industry is based on concepts from the biotech drug manufacturing process, its end goals and product are in fact the exact opposite of the goals of a typical biotech drug substance plant.
The goal of the drug substance plant is to keep the cells in suspension and prevent the cells from differentiating and attaching to each other, but when it comes to meat (food), the goal is to get the cells to differentiate into muscle cells and grow together and bulk up into muscle tissue.
In drugs, the cells are separated from the extracellular desirable proteins that are in the cell culture fluid and are disposed of as waste; whereas in meat (food), the cells are the product, and the cell culture fluid is discarded or reclaimed.
Further, in the food industry, many unit operations involved in seeding scaffold structures and growing meat on those structures are new and have never been implemented at a production scale before. That’s why when one client called on E Tech Group to help with their pilot cultivated meat plant, we knew we had to answer the call with our best, most creative and critical thinking foot forward.
The Project: A Turnkey Production-Scale Automation System for Lab-Grown Meat
The client, one of the first to receive FDA approval to grow and sell cultivated meat for human consumption, came to E Tech Group because of our extensive experience in the biotechnology industry. The client was scaling up from lab-scale benchtop processes to full-scale production with the first-of-its-kind pilot plant to prove that they could manufacture cultivated meat at scale.
At the time, the client was a startup company with limited resources in both personnel and budget. They had no automation staff when we first started working with them and they needed to move quickly from benchtop lab scale to pilot plant scale, and eventually to large-scale production.
This endeavor was critical to the start up’s success; this first-of-its-kind food automation system would essentially be their proof of concept. The client was looking for ease of implementation and benefits, and was especially concerned about the cost-benefit ratio of a complete control system build.
The Approach: Cultivating Consistency at Every Step of Production
E Tech Group was brought in at the conceptual phase of the project and assisted in process design decisions. Our automation engineers supported the client with preconstruction design studies, including electrical design, user requirement specification and functional specification. This included definition of device states for each process step and documentation of flow paths in the form of Pin Charts – a spreadsheet that shows which valves need to be opened and closed for each flow path in the P&IDs. E Tech Group’s input also included process design, phase description and parameter definition.
The cultivated meat processes include:
- Media Prep
- Steam in Place (SIP)
- Bulk Media Fill
- Inoculation
- Cell Culture
- Media Feed
- Seeding and Growth
- Harvest
- Clean in Place (CIP)
Interestingly, despite the opposing goals of drug vs food cell culture, out of the processes named above, only Seeding and Growth differ from traditional biotech cell culture processes.
The customer chose to make the plant as fully automated as possible because they intend to mass produce meat in plants around the world. The product must look, feel and taste the same, and should have the same consistency and texture no matter where it is produced.
Automating food manufacturing processes accomplishes this by creating consistency and repeatability. Automation also allows for quick and easy recipe generation and recipe changes based on new and different products, as well as new discoveries in the research and development department. A fully-automated pilot plant would demonstrate the potential for flawless, consistent cultivated meat production in a framework that allows for adaptability, flexibility and scale.
The Solution: Applying Rockwell Automation’s PlantPAx Platform
Rockwell PlantPAx was selected for this project because it had many advantages over other solutions. The plant had aggressive programming milestones and an evolving process design that would require on-the-fly changes. The client wanted to use a product that could be rolled out to many plants across the US and eventually across the world, without the concern for the availability of local automation support wherever they build. As one of the most open and widely used control systems in the United States, Rockwell fit this bill.
Rockwell had many advantages with out-of-the-box features including a pretested embedded PlantPAx process library, standard Equipment Module (EM) framework with embedded state machines, and a flexible SCADA solution that could display data from both stick-built, and vendor-supplied standalone skids.
The plant used remote IO panels prewired by the skid vendors from local skid instruments to Rockwell 1794 remote I/O. This way all the wiring could be tested at the skid vendor site prior to shipment. Only the Device Level Ring (DLR) Ethernet IP connections needed to be wired and tested on-site.
E Tech Group’s automation solution was designed to be reusable and scalable. Although the plant started out with only a few bioreactors, more units were planned to be added in the future. For eventual use at full production scale, the modules were designed to be transportable to larger and larger bioreactors. This design included an ISA S88-based modular batch model.
The ISA S-88 programming in Rockwell’s FactoryTalk Batch included five units in the process cell with over 88 phases configured in those units. Recipes, Unit Procedures, and Operations were configured to parameterize and coordinate the equipment phase execution.
For transfers between units, linked groups were created to manage transfer phase communication between source and destination phases. The equipment phases drive 55 equipment modules with 1100 control modules providing inputs and outputs to the batch. Equipment phases were developed and tested based on the Pin Charts.
The Specs: PlantPAx Control Modules & Equipment Modules
The configuration utilized the PlantPAx standard control module (CM) library. This includes control modules for block valves, control valves, pumps, variable frequency drives, analog inputs, digital inputs, and PID loops. The process equipment to be controlled included media prep vessels, portable tanks, transfer lines, small bioreactors, large bioreactors, centrifuge, and Portable CIP skid.
The Studio 5000 Application Code Manager (ACM) bulk import/export tool was used to import the I/O list and configure the control module attributes. The project used a mix of PlantPAx 4.10 Add-On Instructions (AOI) library elements and embedded PlantPAx 5.0 function blocks. The configuration used both function block and ladder logic-based control and equipment modules.
Equipment modules were created to control and coordinate groups of valves and control elements that must work together for a specific function. The equipment modules are commanded to control schemes by phase logic. Within the Equipment module is mode propagation logic to ensure all subordinate modules are in the correct mode.
The transition from scheme to scheme puts the subordinate control modules in the correct state for the desired scheme and may sequence the changing valve states in a configured order to prevent water hammer or other undesirable issues during the transitions. The EM also aggregates alarms, permissive, and interlocks for the subordinate CMs and propagates response to those abnormal conditions by transitioning to a safe scheme.
Equipment modules were used for the complex control loops that control the bioreactor environment. These custom EMs included:
- Temperature Control
- 8CM
- 4 Schemes
- Pressure control
- 9 CM
- 9 Schemes
- Dissolved Oxygen (DO) control
- 19 CM
- 16 schemes
- pH Control
- 6 CMs
- 6 schemes
- Overlay Air Control
- 7 CM
- 8 Schemes
- Agitator control
- 4 CMs
- 5 schemes
- Feed control (Antifoam, Nutrient, and Chemical feeds)
- 3 CM
- 4 schemes
- Mix Proof Aseptic valve control
- 7 CM
- 12 Schemes
The mix-proof valve is a good example of the complex programming involved in combining standard PlantPAx control modules together with the standard EM construct to create an equipment module that can be reused for every mix-proof valve in the system. The mix-proof valves are used to separate the media loop and the harvest loop from the bioreactors. This enables the cleaning of the loop while the product is the bioreactor – without the fear of cross-contamination of the bioreactors with the cleaning solution. Hence the term “Mix Proof”. The valve is physically separated into 3 chambers – Top, Middle isolation chamber, and Bottom.
The control of one mix-proof valve consists of 7 automation elements:
- 4 Valve actuators
- Main valve
- Steam Inlet valve
- Chamber Drain Valve
- Steam Trap divert valve
- 2 Seat lifters
- 1 Temperature input
The mix-proof EM has 12 schemes:
- Mix Proof
- Flush
- SIP Chamber
- Cool Down
- Close Steam
- Open (Sanitary)
- CIP Top
- CIP Bottom
- CIP
- SIP Top
- SIP Bottom
- Open
To make a sanitary transfer the EM must sequence through schemes 1 through 6 in order.
ISA S88 Batch Management
Equipment phases were configured in the controller using structured text for the Running, Holding, Restarting, Stopping, Aborting, and Resetting sequences. Phase parameters and report parameters were created for each phase.
The phases use the standard PlantPAx Prompt module to display operator prompts to operators. This module can also request data to be entered manually by the operator, which in turn is included in the phase report parameters.
The phases also communicate with external skids such as the portable CIP skid to keep the phase in sync with the sequence that is running in the external skid. A FactoryTalk Batch Equipment Model was configured by importing the equipment phases. FTBatch linked groups were created to facilitate messaging and handshaking between phases that need to communicate with each other.
FactoryTalk View Site Edition SCADA
The pilot plant SCADA is compliant with 21 CFR part 117 the FDA standard for Electronic Records and Electronic signatures in the production of meat products. PlantPAx comes with graphical elements symbols and face plates based on the ISA 101 standard SCADA package used with FactoryTalk View SE version 12.
The SCADA solution features remote unit operations, centralized data monitoring, process data storage, and trends. The alarm management based on ISA 18.2 was enhanced to provide easy filtering of safety alarms versus product quality alarms. The system included remote alarm notifications to targeted personnel. FactoryTalk alarm builder was used to facilitate the development and management of the alarm severity and messaging.
The server infrastructure is a VMware-based virtual environment hosted on enterprise-grade hardware hosts that are maintained by the client’s corporate IT. Prebuilt virtual templates provided by Rockwell were used to build the virtual servers.
Servers include:
- Network Directory Server
- Domain Server
- SQL Server
- SCADA Server
- Terminal Servers
- Historian Server
- Batch Server
- AssetCentre Server
- Engineering Workstation
- Win 911
To minimize the need for support staff on the plant floor a thin client terminal server architecture was used where the thin client has no Windows operating system or other software that needs regular updates. The SCADA network used ThinManager ready thin clients for HMI stations displaying graphics on off-the-shelf desktop monitors to minimize costs. Rockwell’s ThinManager allows for all thin clients to be managed from a central location off the plant floor.
When a thin client comes online or is replaced, it locates the ThinManager and the ThinManager provides the client with its configuration – including installing monitor drivers if required, and the Terminal server application to open on the terminal server and default HMI screen(s). If a terminal server fails, the ThinManager can have the thin client seamlessly switch to a backup server.
ThinManager can also load balance between multiple terminal servers – sending the thin client to the most lightly loaded terminal server. ThinManager also allows for a shadow session where an engineer can watch the session on the floor from a remote location. The shadow session is very useful for troubleshooting.
Alarms are based on the ISA 18.2 standard. Node-based security was implemented which identifies the node location when the operator is logging into the system from. Knowing the location allows the system to serve graphics and filter alarms that apply to the equipment near the terminal. Location can also be used to restrict critical operator interactions to the terminals that are near the process.
OEM Skid Integration
Other systems were tightly integrated into the control system. This included the Hazard Monitoring System (HMS). Comprehensive interfaces were developed for external skids that needed to work in lockstep with the stick-built equipment. The centrifuge skid and the portable CIP skid were tightly integrated into the control system with handshaking at critical points in the process.
External standalone skids that used PanelView View ME screens were imported into the system so that they could be viewed from the SCADA package and data from those systems could be forwarded to the historian. There were also custom graphics created for external skid integration.
FactoryTalk Historian
Accurate collection and storage of historical data is necessary to meet regulatory requirements. Analysis of historical data is also essential in a new process so one can see the effect of changes made to the process on the quality attributes of the final product. The plant used FactoryTalk Historian for the data historization package.
FactoryTalk historian is an OEM version of OSI PI; this allows the plant to eventually add a corporate-level OSI PI layer above all their plants to aggregate data for the entire company.
Remote Alarm Notification
Remote alarm notification is critical to the pilot plant as the plant is capable of operating 24/7, and there are limited personnel onsite during off-shift hours. Win 911 was used to capture and forward alarm messaging to targeted personnel. Remote alarm notification is also required for regulatory compliance.
Audit Trail & Reporting
The system was delivered with FactoryTalk AssetCentre to minimize paper audit trail changes and handwritten logbooks. AssetCentre is used for controller version control, system audit trail, and system backup and restore. AssetCentre enables an audit trail of the operator interactions on PanelView HMIs.
Rockwell has canned reports for use with FTBatch. These include batch list, batch summary, batch details, material usage, forward tracking, backward tracing, batch execution, duration compare, batch exception, and equipment use log. Pre-built queries can be edited to customize reports like Dream Report.
The Results: Proof of Potential for Full-Scale Cultured Meat Manufacturing
The system was fully factory-tested before delivery to the client. This testing used the virtual simulation capabilities of the PlantPAx library elements and the Logix Echo emulated process controller. The Echo is a software-only emulated version of the hardware PLC delivered to the site. The Echo can run the exact same configuration software as the hardware version with no changes required.
The project team adapted to an evolving design and developed a dynamic and flexible solution as the pilot plant was being built. The project was a success with the site ready for production in less than six months. The completed pilot plant is a showcase facility where investors can learn about the cultured meat industry and even sample the cultured meat product in the facility’s test kitchen.
New projects are ongoing, including a corporate PI layer in anticipation of a proposed full-scale manufacturing facility. E Tech Group looks forward to continuing our foray into this new frontier of food production automation with this innovative client.