Helping a Food & Beverage Client Avoid Obsolescence Concerns
A large food and beverage company recognized there was the potential large-scale obsolescence of their control systems. They reached out to E Tech Group to have our automation team assess and provide plans for remediation and optimization.
The Project: Address the Possibility of Obsolescence Ahead of Time
A large food and beverage client reached out to us, as they were concerned about the state of both their ethernet and non-ethernet connected Industrial Control System Networks at all thirty-six facilities. Understanding there was an inherent risk of obsolescence as the state of their network, physical asset health and security was of an indeterminable nature, they wanted to have an IT/OT Vulnerability Assessment done to identify any potential threats, what was nearing end of life, and areas where they could push for optimization. They needed an in-depth assessment and a targeted plan for remediation.
Having an existing working relationship with this client, E Tech Group was immediately one of the control system integrators considered for this project, and was eventually contacted for a bid. If awarded this contract, E Tech Group would be responsible for walking all thirty-six of their facilities to perform the same type of assessment and provide a plan for remediation.
The Process: Assess, Diagnose & Remediate Control System Concerns
After reviewing proposals, the client selected E Tech Group and another control system integrator to provide the initial assessment and develop their individual plans for remediation based on their findings. We knew our IT/OT assessment and subsequent project design needed to be competitive and cutting-edge while remaining attainable.
This assessment would cover both a network driven and physical on-site inventory and evaluation process. Once completed, the findings would be rolled up into reporting that would be consumed at the corporate level down to the operations team.
For the network portion of the assessment, the client’s IT provided our team documentation around program files for the PLCs, HMIs, Drives, and were granted network vizios to view network topology. They were then given limited network access, and using different protocols were able to show where their PLCs, HMIs and Drives were connected on their IT network. This allowed them to dig a bit deeper into their topology rather than just where their IT switches were.
Lastly, the E Tech Group team worked to compile an asset inventory to help the client understand what equipment was currently living on their network and where in its lifecycle it was. This was a two-prong process that required close collaboration between our automation experts and the client’s operations personnel to ensure that all bases were covered.
Network Assets to be inventoried and assessed:
- Network switches (Not including Plant Network IT devices, For example Cisco)
- Routers
- Firewalls
- Servers
- PCs
- NATs
- HMIs
- PLCs
- VFDs
- IO Racks
- Any other Devices on a communication network
With it now time to walk the facility floor, our E Tech Group team members would be joined by facility engineers, technicians and other staff to assist in locating the critical components that needed to be identified, documented and inventoried. Working closely with their on-site team allowed this process to move more efficiently and ensure nothing was overlooked.
Our automation and IT engineers utilized the existing controls network logical infrastructure drawings to work swiftly, documenting and inventorying each panel and its contents, making sure to note if they were reflected accurately in the provided drawings.
Physical Assets to be inventoried and assessed:
- Physical Network Infrastructure, for both ethernet and non-ethernet.
- Physical Topology and Active Device Inventory
- Switch Selection (Not to include IT Infrastructure Cisco Switches)
- Router Selection (Not to include IT Infrastructure Cisco Routers)
- Communication configurations for both ethernet and non-ethernet (Configuration, Lost Packets, Collisions, etc)
- Environmental Conditions
- Enclosures
- Cable Selection
- Cable Management
- Conduit and Routing
- Cable Labeling
- Power Redundancy
- Grounding
Once all digital and physical assets were documented, the E Tech Group team worked diligently for two weeks to take this massive aggregate of information and transform it into a usable database for the client to digest.
The Solution: Tailor Remediation Plans to Each Facility’s Needs
Upon initial review, the client was painfully unaware so many components were near end of life or patchworked together to keep them running. This targeted view into their IT and OT helped them prioritize their next steps that would allow them to start working more proactively to improve and maintain their network and asset health.
Because the E Tech Group assessment best aligned with the client’s standards, we were awarded the contract. Using this initial work as a benchmark, the templated assessment would now be scaled across all thirty-six sites, allowing for tailored plans of remediation.
The Results: Avoided Complications of Large-Scale Obsolescence
Long term, the client intends to set themselves up to implement a 3D model to monitor their network health and asset lifecycle. This system would allow them to utilize predictive maintenance for planned downtime that has the least impact on production, improving efficiency and uptime as upgrades are performed. Understanding the client’s goals, E Tech Group outlined their first plan for remediation:
Site 1 Phases of Remediation:
- Consolidate network servers across all facilities
- Identify isolated assets not living on the network
- Work to migrate non-network connected assets
- Implement connected shop floor cloud-based monitoring to track asset efficiency and network health
The E Tech Group team will continue to work through the remaining sites, utilizing the templated assessment to help build customized plans for remediation. Since these sites were acquired individually and over time, there is a wide range of equipment manufacturers and assets in various stages of their lifecycle.
Each facility will require a different type and extent of control system upgrade, which is why the assessment design was key in this process. Being able to re-use our assessment strategy across all sites will help in quickly identifying vulnerabilities and being able to move into the remediation process as quickly as possible.
Begin Your Journey to Industry 4.0
Download our Beginner’s Guide to IT/OT Assessments to gain an understanding of what an IT/OT Assessment is, its benefits, and how to get started today.
Challenge
Outdated controls can pose many operational challenges. Companies that operate with outdated controls often experience frequent downtime, expensive or obsolete hardware replacement parts, or few personnel with knowledge to perform maintenance on legacy systems. These challenges were faced by a long-running California winery who needed to modernize the controls for their red fermentation process. The winery needed to address how a system upgrade would impact their operations and engaged E Tech Group to create a PLC and SCADA upgrade solution.
Solution
The first step was to create a well-documented project plan. E Tech Group analyzed the existing system design and detailed existing hardware components, software configurations, and network connections. This step was crucial to have an accurate picture of their current control environment to mitigate production impacts during the implementation phase.
The project team upgraded hardware and reconfigured the customer’s existing Ignition SCADA software for seamless post-upgrade operations. The original red fermentation process was conformed with three areas: pump over area, continuous press area, and decanters area. Previously, each area had their own PLC but the upgraded system connected an Allen-Bradley ControlLogix PLC as the main controller and remote IO was added to control the three areas into one. The system was programmed with ladder logic to the customer’s existing Ignition SCADA software, and seven new PanelView Plus HMIs were also programmed with FactoryTalk View ME as the customer’s choice visualization software.
Results
The project team leveraged their PLC programming, Ignition SCADA software, and controls engineering expertise to resolve software compatibility issues, bugs in the programming code, and electrical connection challenges. The team supported the project to resolve these challenges and performed extensive testing. With the PLC and SCADA upgrade complete, the customer was able to improve the uptime of its red fermentation process and eliminated the key challenges of legacy system maintenance.
Challenge
Complex upgrades require advanced expertise. Experienced system integrators will use a holistic approach to fully scope automation system upgrades and consider how the project’s complexities will impact the customer’s budget, uptime, timeline, or performance requirements. An example of this was a PLC-5 and SCADA application upgrade for a leading rice milling company. The company had an obsolete PLC-5 controlling their rice finishing line, in addition to an obsolete SCADA application controlling their entire facility. The legacy SCADA system was costly to keep operational, and the existing network caused communications issues between the customer’s production areas. The customer needed a new PLC program to be installed within a 48-hour timeframe to minimize production impacts, and the existing SCADA system needed to remain operational during the migration to Inductive Automation’s Ignition platform. E Tech Group was engaged as the customer’s choice system integrator given their advanced expertise in system migration and the successful execution of projects at the customer’s other facilities.
Solution
E Tech Group’s goal is to set the customer up for long-term success, not just short-term gains. The project team created a site inventory of the customer’s existing PLCs, network, computers, and applications. PLC specifications were quickly defined, and long lead items were ordered. The project team remained in constant communication with the customer to keep them well-informed across the different project stages and how to best utilize the new system. The project included upgrades to servers, computer, and network switches in collaboration with the customer’s IT department, the upgrade to Ignition SCADA software across six applications with approximately 500 screens, and the upgrade to two ControlLogix PLC chassis.
Results
The SCADA upgrade was completed in several phases and each application was upgraded individually. This helped minimize impacts to the customer’s production efforts and enabled the legacy and new SCADA applications to run simultaneously so the customer’s personnel could learn the new SCADA software. E Tech Group staffed the project with subject matter experts in the Ignition SCADA platform, control engineering, and complex system migrations. E Tech Group leverages this diverse, multidisciplinary engineering approach to create intelligent automation solutions for the customer’s long-term success and the opportunity to support their operations in the years to come.
Challenge
A supplier of Latin food products was challenged to increase the efficiency of their canned goods sterilization process. Their goal was to move away from a manual sterilization process and move into the future with automation. The food products supplier engaged E Tech Group to improve the efficiency, accuracy, and the flexibility of their operations given their in-depth experience in automating manual processes.
Solution
The first step was developing a solid system design. E Tech Group’s project team built the controls scheme from the ground up with considerations specific to the customer’s processing needs and sterilization standards. These considerations included hardware reliably, accurate instrumentation, and logical program flow.
The automated process design leveraged the customer’s existing controls and integrated new technologies. This included an Allen Bradley CompactLogix Controller, Flex IO Remote Rack, PanelView HMI, ABB Drives, safety monitoring relay, and instrumentation for high temperature environments. The detailed system design allowed for a smooth installation and commissioning of the project.
Results
E Tech Group staffed the project with subject matter experts in instrumentation, Ethernet-enabled devices, temperature control, safety equipment, and electrical design to support a successful execution. The project team designed, developed, tested, and commissioned the project in about six weeks and helped the customer reach their goal of increased efficiency. The automated sterilization process helped increase the customer’s production rates in comparison to their previous manual process, and provided enhanced data reporting to make informed production decisions. With the automated sterilization process in place, the customer is able to improve the productivity, accuracy, and flexibility of their operations.