Project

A large food and beverage client reached out as they were concerned about the state of both their ethernet and non-ethernet connected Industrial Control System Networks at all thirty-six facilities.  Understanding there was an inherent risk of obsolescence as the state of their network, physical asset health and security was of an indeterminable nature, they wanted to have an ITOT Vulnerability Assessment done to identify any potential threats, what was nearing end of life, and areas where they could push for optimization utilizing a targeted plan for remediation.  Having an existing working relationship with this client, E Tech Group was immediately one of the integrators considered for this project and was contacted for their bid.  If awarded this contract, E Tech Group would be responsible for walking all thirty-six of their facilities to perform the same type of assessment and provide a plan for remediation.

Assessment Process

After reviewing proposals, the client selected E Tech Group and another vendor to provide the initial assessment and develop their individual plans for remediation based on their findings. 

This assessment would cover both a network driven and physical on-site inventory and evaluation process.  Once completed, would be rolled up into reporting that would be consumed at the corporate level down to the operations team.

For the network portion of the assessment, the client’s IT provided our team documentation around program files for the PLCs, HMIs, Drives and were granted network vizios to view network topology.  They were then given limited network access and using different protocols were able to show where their PLCs, HMIs and Drives were connected on their IT network so they could dig a bit deeper into their topology rather than just where their IT switches were. 

Lastly, the E Tech Group team worked to compile an asset inventory to help the client understand what equipment was currently living on their network and where in its lifecycle it was. 

Network Assets to be inventoried and assessed:

With it now time to walk the facility floor, our E Tech Group team members would be joined by facility engineers, technicians, and other staff to assist in locating the critical components that needed to be identified, documented and inventoried.  Working closely with their on-site team would allow this process to move more efficiently and ensure nothing was overlooked.  The E Tech Group team would utilize the existing controls network logical infrastructure drawings to work swiftly, documenting and inventorying each panel and its contents, making sure to note if they are reflected accurately in the provided drawings. 

Physical Assets to be inventoried and assessed:

Now that all digital and physical assets have been documented our team would work diligently for two weeks to take this massive aggregate of information and transform it into a usable database for the client to digest. 

Solution

Upon initial review, the client was painfully unaware so many components were near end of life or patchworked together to keep them running.  This targeted view into their IT and OT helped them prioritize their next steps that would allow them to start working more proactively to improve and maintain their network and asset health.  Because the E Tech Group assessment best aligned with the client’s standards, we were awarded the contract.  Using this initial work as a benchmark, the templated assessment would now be scaled across all thirty-six sites, allowing for tailored plans of remediation.

Results

Long term, the client intends to set themselves up to implement a 3D model to monitor their network health and asset lifecycle to utilize predictive maintenance for planned downtime that has the least impact on production, improving efficiency and uptime as upgrades are performed.  Understanding the client’s goals, E Tech Group outlined their first plan for remediation.

Site 1 phases of remediation:

• Consolidate network servers across all facilities
• Identify isolated assets not living on the network
• Work to migrate non-network connected assets
• Implement connected shop floor cloud-based monitoring to track asset efficiency and network health

The E Tech Group team will continue to work through the remaining sites, utilizing the templated assessment to help build customized plans for remediation. Since these sites were acquired individually and over time, there is a wide range of equipment manufacturers and assets in various stages of their lifecycle.  Being able to re-use their assessment strategy across all sites will help in quickly identifying vulnerabilities and being able to move into the remediation process as quickly as possible.