We’re your source for automation news. Keep up with the latest industry updates and E Tech employee spotlights, as well as tips and guidance from our manufacturing experts.
Upgrading an Irrigation District Water Treatment Plant’s Control System
E Tech Group upgraded a decades’ old, primitive control system in an irrigation water district treatment plant to allow for remote monitoring and control of some of their systems, more reliability, and less maintenance.
The Project: Modernize an Antiquated Control System for a Water Treatment Plant
An irrigation district water treatment plant’s components were reaching end of life from their original installation E Tech Group executed almost thirty-years prior. The relationship built and maintained over those years, coupled with a quality system kept running with standard maintenance and a recent upgrade to their SCADA System computers and software allowed for E Tech Group to step right back in to retrofit and upgrade this facility’s control system. Our team worked from a Design-Build approach, working directly with the client to develop the scope of work and design:
The Scope: Drowning in Problematic End-of-Life Symptoms
The team was asked to tackle the upgrade of eleven control panels. The main plant’s PLC CPU was an obsolete Rockwell L61 that needed to be replaced with a current model, firmware and software. The main driver for this section was the fear of equipment failure and risk of extended plant downtime.
The raw water pump station and washwater panels used obsolete Rockwell L34 PLC CPUs which had recently begun to degrade to the point they began losing programming at random times and also required frequent reprogramming. This was a serious inconvenience to the operations staff as there was little to no predictability of these outages, which meant the problem could arise at any time, day or night.
The only current solution the client had found was to run manually for an extended period of time until the issue was temporarily resolved, which required an increase in maintenance labor and cost. The plant remote I/O panels were connected via ControlNet and the client wanted to move to EtherNet/IP providing a more stable and reliable connection. A need for redundancy in the main plant PLC and network was identified, leading to the placement of the remote I/O in a ring topology.
Wanting to implement more remote monitoring, the decision was made to add remote control and monitoring for the plant air compressor system as the existing was local control only. The last piece identified for this upgrade was to remove two obsolete physical PID controller devices from the main control panel and replace them with PLC software based PID controllers.
The Challenge: Maintain Uptime During a Control System Upgrade
Before we could begin executing design and installation with the defined scope of work, there was one large restriction that would require a creative solution to navigate. The plant would need to remain operational during the retrofit. The team would be allowed downtime for each panel, but not extended downtime for the entire plant.
To maximize efficiency our team modified the PLC program to allow them to maintain a hybrid system containing both the ControlNet and EtherNet/IP remote I/O modules. With this modification they were able to keep downtime to less than an hour per panel for the remote I/O conversions.
An environmental challenge that no one could have accounted for during planning was that a mouse had chewed through one of the fiber-optic links between control panels. This forced the temporary reuse of the existing Ethernet radios to keep the remote I/O rack online and functional until the fiber-optic link could be fixed.
The Solution: Define, Design & Install a New Wastewater Automation System
With the scope of work now defined, potential challenges identified and accounted for, our automation system integrators would move forward in the development and installation of a solution that would move this facility into the twenty-first century with increased reliability, decreased downtime and lower maintenance costs.
E Tech Group’s custom control system solution included:
Designed a new Ethernet and Fiber Optic network topology.
Developed a retrofit design package detailing the modifications that we were going to make to each panel.
Developed a retrofit commissioning plan detailing the process and sequence of how we intended to modify the system.
Migrated the PLC program from version 16 to 30, migrated the L34 CPU programs into the main PLC program, added the plant air and PID programming.
Platform – Upgrading the system with newer Rockwell components made the most sense as the programming could be migrated, and the existing I/O modules could be reused.
Replaced Rockwell L61 CPU with Rockwell L81 CPU and upgrade firmware/software – v16 to v30.
Replaced Rockwell ControlNet communication adapters with Rockwell Ethernet/IP communication adapters.
Replaced Rockwell ENBT Ethernet modules with Rockwell EN2TR Ethernet module.
Replaced Rockwell CNB module with Rockwell EN2TR Ethernet module.
Replaced Rockwell L34 CPU to EtherNet/IP remote I/O conversion – moved logic to the main plant PLC
Partnership – Installation of new Ethernet and Fiber Optic infrastructure
The client self-performed the following work under the guidance and direction of E Tech Group
Installed conduit needed for the new network infrastructure.
Installed Ethernet and Fiber Optic cabling between panels needed for the new network infrastructure.
Partnership – Commissioning
Operations worked closely with E Tech Group during commissioning to keep the plant running during each panel cutover.
The Result: Retrofit Increases Reliability & Grows Relationship
Our team was able to deliver the proposed solution on time and on budget, all while keeping the facility operational, aside from the agreed upon downtime for each panel. The plant is now running optimally using current automation products, along with an inventory of spare parts onsite should something happen, reducing the risk of long-term downtime significantly.
The system is more reliable now that the L34 PLCs have been removed; no more middle of the night surprises. Operations is now able to monitor and control the plant air system, eliminating the need for manual operation and consolidating resources to a more central location. The client was incredibly happy with the quality and proficiency of our work and now allows us to list them as a reference.
Outdated systems have the potential to transform assets into liabilities. It is important to know the signs and risks of using outdated systems, and when it is the right time to invest in a modern solution. This was a challenge faced by a company specializing in environmental management services who was operating eleven water well sites across a five-mile area. These sites were being controlled by legacy PLCs and HMIs and relied on slow serial radios for communication, posing several risks such as high maintenance costs, poor HMI responsiveness, and limited scalability due to the outdated equipment. The customer recognized how their outdated system negatively affected their productivity and efficiency which prompted them to seek engineering support. The company engaged E Tech Group to design and deploy a modern solution, which included upgrading their PLC, HMI, and radio controls as well as install an additional water well to expand their operations.
Solution
Before installing the new water well, the project team focused on updating the customer’s existing hardware which included the PLCs, HMIs, and radio controls. E Tech Group partnered with Applied Technology Group, a company specialized in radio controls, to investigate how to implement a modern Ethernet communications protocol for the customer with minimal disruption to their operations. The research concluded that the best order to implement the upgrades was in two phases. The first phase was to upgrade the Access Point (AP) radio at the customer’s control office, and then upgrade the remote radios that communicate to the AP radio and the main control PLC. With this information, E Tech Group began developing the updated system.
The project team upgraded 13 legacy SLC controllers and Panel Views for newer CompactLogix controllers and Panel View 7 Plus terminals, using Studio5000 and FactoryTalk View Studio to program them. The application also integrated Inductive Automation’s Ignition Perspective SCADA system which provided an operation and management overview of the wells status to the customer’s local and corporate offices.
The project team set up an intermediate infrastructure at the customer’s control office that could run the water well upgrades in parallel with the legacy system. The team installed the new PLCs to run concurrently with the legacy PLCs, and configured messaging between the controllers so the main legacy PLC was unaware of the upgrades.
The team then upgraded the customer’s networking communications from serial radios to Ethernet radios with a more powerful antenna. Lastly, the project team decommissioned the legacy PLC and converted the water wells to run exclusively on the upgraded system.
Throughout this process, E Tech Group also resolved challenges with the customer’s remote water wells which faced poor connections from antenna height limitations. The team implemented a store-and-forward capability for the remote radios, which transmitted data to a closer radio station and then forwarded to the main AP radio to improve data connectivity.
After modernizing the existing water wells, E Tech Group installed and commissioned the new water well.
Results
The project, including the design and control panel construction for the new water well, was completed in just a few short months. E Tech Group successfully supported the modernization of the customer’s outdated system by assigning subject matter experts in network design, radio configuration, PLC/HMI coding and conversions, and the Ignition Perspective module to the project. The implementation of the new system resulted in significant enhancements for the customer in terms of speed and reliability. They now have real-time data available on a per-second basis instead of waiting for minutes, allowing them to depend on it more effectively. Moreover, the customer managed to expand their existing infrastructure by integrating the new water well, effectively meeting their growing demands and overcoming the challenges posed by their previous outdated system.
Challenge
SCADA systems play an important role in production processes. Over time, these systems can become unsupported or lack the same level of functionality of new SCADA systems. This was the challenge for a company specialized in environmental management services including water pollution. The company’s existing HMI software and server operating system were at its end-of-life status which impacted the visibility of their groundwater remediation processes. For example, the existing HMI application only allowed a single remote desktop session into the server through a legacy VPN solution, which was a challenge as many of their operators worked in remote offices or in the field. The company chose to migrate to Inductive Automation’s Ignition platform with the Perspective Module as its web-based interface would enable multiple operators to monitor and manage processes from any mobile device. E Tech Group was engaged to support the Ignition SCADA migration which consisted of upgrading the HMI software and servers, networking equipment, and VPN solution.
Solution
The first step was developing a plan that reduced migration risk and increased visualization flexibility. E Tech Group created a network diagram and planned for the customer’s operators to predominantly use the new HMI software and servers during the transition period. The legacy software was also available to reference if needed. The new networking solution included considerations to mitigate the facility’s frequent power outages and incorporated a new uninterruptible power supply (UPS) with network diagnostics in addition to Ignition Alarming functionality. This would alert operators of a power outage and automatically perform a controlled shutdown of the SCADA server, supporting a longer server lifespan and increased data protection.
The system was programmed to automatically restart and resume its operations once the power was restored without personnel needed to be on-site. As a custom solution, E Tech Group leveraged virtualized Windows-based servers for the operating system, Ignition Perspective for the SCADA platform, an open-source SQL database to house the historical process data, and an industrial grade VPN solution to support secure process monitoring and control for remote personnel.
Results
The design of the new SCADA platform was completed in a month, with two weeks of development for additional features as requested by the customer. The final design was a collaboration between the customer and E Tech Group who staffed the project with subject matter experts in the Ignition platform, server hardware and configuration, and SCADA programming. The new design enhanced the customer’s HMI features and supported mobile-optimized, real-time reports to control their environmentally conscious processes across the customer’s remote locations.
E Tech Group Lead Automation Engineer, Kevin Tom shared his knowledge and expertise on Best Practices for Effective Automation Applications for Control Engineering Magazine‘s webcast on August 16th, 2022, where he discussed the nuances of control system applications. When Should a Process be Automated? Kevin explains the criteria that a company should consider when weighing whether or not to apply automation to a process: What is being made and how is it currently being produced? Consider tools, parts, mechanisms. What is the environment like? Consider space constraints, equipment, hazards. What is the labor pool like? Consider union/non-union, labor shortages, turnover. In considering automation, goals and constraints must also be hashed out: Are we trying to replace human labor with mechanized processes? Are we trying to scale up operations/increase throughput? What is the capital we have to/are willing to invest in this project and what is the necessary ROI? These questions are all addressed early on in a project’s inception, as stakeholders hold the keys to the budget needed for an automation project. However, control system engineers should also be consulted early on to give insight into the specifics of these questions, like cost, ROI, improvement caps. Applying Automation is as Much Planning as Execution Kevin Tom’s cohost points out that much of the time, the options are: automate or die. It may sound histrionic, but the labor shortage is a real and pervasive issue, especially when manufacturers are trying to scale. Increased production goals means hire more people. But there aren’t enough people to be found, and/or high turnover, and/or lack of qualified professionals, which all present labor bottlenecks. Automation is the best (and only) solution to avoiding the obstacles associated with labor shortages. However, it isn’t as simple as design-build-implement. For the transition to automation to run smoothly, the system first … Continued
