We’re your source for automation news. Keep up with the latest industry updates and E Tech employee spotlights, as well as tips and guidance from our manufacturing experts.
Integrating an Electrical Power Monitor System in Parallel with an Existing System
E Tech Group installed and commissioned a standard Electrical Power Monitoring System (EPMS) in parallel with a data center’s existing system to maintain the original combined EPMS and Building Management System (BMS) until it could be upgraded.
The Project: Install a New EPMS in Tandem with an Existing EPMS & BMS System
A social media platform Data Center needed their standard Electrical Power Monitor System (EPMS) installed and commissioned to work in parallel with their older existing system. The existing system was a combined EPMS and Building Management System (BMS) and needed to remain in place, as the BMS portion of the system wasn’t being replaced yet.
The multi-building Data Center campus and the SCADA system in some of the buildings is the current standard SCADA software, but in other buildings, older SCADA software was in place from a different vendor needing alignment.
The client wanted to eliminate device connections utilizing the BACnet/IP protocol and replace them with Modbus/TCP protocol to ensure current system standards are being met. As well, they wanted to standardize points being polled and monitored to match the current standards of the newer sites within the data center fleet.
E Tech Group’s reputation within the client’s organization, performance on a prior project, and because our feasibility study gave the client confidence that the project had a high level of probability that it could be accomplished within the limited timeframe allowed, we were selected to complete this project.
E Tech Group’s automation team knew they needed to provide continuity to rectify the impact the client was currently experiencing as well as become familiar with and operate within two systems instead of just one. The aggressive timeline coupled with multiple moratorium periods to work around required an incredibly strategic and efficient approach.
The client was concerned that it wasn’t feasible to have an additional system polling for data in parallel and the equipment wouldn’t be able to support the additional requests. Some of the equipment was already being polled by 3-4 other systems monitoring the data. E Tech Group:
Performed a feasibility study/risk assessment as an initial phase of this project.
Evaluated each device type to determine limitations and assess which devices might experience bandwidth, connection limit, and response time issues.
Developed a plan to mitigate the issues.
The BACnet/IP communication modules in the generator control panels needed to be replaced with Modbus/TCP communication modules. The design of the panels made it necessary to perform a Lock Out – Tag Out (LOTO) of each generator to install the new communication modules safely. We worked with site Operations and developed a schedule to perform the work, aligning as much as possible with their planned maintenance activities.
The Solution: Gain the Client Confidence Needed for Our Team to Divide & Conquer
The Feasibility Study mentioned earlier provided confidence in the proposed solution, and its approval was a catalyst for team members to move forward knowing they had a tight timeline they were up against.
To divide and conquer, E Tech Group subcontracted the SCADA developer required by the customer and managed the development of the EPMS SCADA application.
PowerSCADA was the software platform used as the client’s standard.
It was developed in phases in order to meet the project schedule, which required tracking changes with each version and verification to ensure the final application incorporated all of the necessary edits.
Our team performed bench testing of each field equipment type using a Modbus simulation device to verify the graphic templates and tags were linked as expected.
We developed a commissioning plan and corresponding method of procedures for bringing the new system online. To avoid impacts to the live system our team tested one device type at a time and monitored the health of the existing system, ensuring the two were able to function together without interruption or conflict.
Onsite and remote commissioning of the SCADA system and the data center infrastructure management (DCIM) interface.
The Result: Active Partnerships Create the Most Effective Control Systems
By partnering with the client, as opposed to just fulfilling a role or requirement, E Tech Group helped the customer get closer to their long-term vision of the system. The new EPMS system follows the standards used at other sites within the data center fleet.
The findings of the feasibility study proved to be accurate, and the information gathered allowed the system to be successfully commissioned without any major impacts to the live system.
The client stated they were incredibly satisfied with the way the project was executed during the lessons learned meeting at project completion. This successful commissioning of the EPMS SCADA application allowed the client to be ready for the upcoming project of integrating the Building Management System.
Secure Your Company with a Cybersecure Control System Cybersecurity continues to gain more and more momentum in manufacturing, and for good reason. Some of these facilities can be responsible for millions, sometimes billions of dollars in business, quickly gaining attention and unfortunately becoming a target for a cyberattack if not properly secured. At a time when manufacturing facilities tend to be one of the most impacted by the decline in available workforce, the need for more flexibility in production capabilities takes precedence over most other initiatives. The rush to connect available assets for consolidated control and monitoring, and integrating their data collection and analysis will sometimes push decision makers to leap before they look. Not always considering any of the following, or lack thereof, may become an active threat without proper planning: Increasing IT/OT Convergence: Industry 4.0 & improving plant efficiency is a major driver for this convergence trend. Organizations see improved efficiency as a worthwhile end goal, but it does come with an expanding OT attack surface. High-Profile Incidents: Noteworthy OT/ICS cyberattacks over the last several years have proved how easily these high-profile OT systems can be broken into causing real concern. Some examples are the Stuxnet worm, the Ukraine power grid attack, and the Colonial Pipeline ransomware attack. These examples show that OT/ICS incidents affect more than the targeted organizations. Nation-State Threats: Nation-state actors have shown interest in targeting critical infrastructure and industrial sectors as part of cyber espionage or cyber warfare efforts. Ransomware and Monetization: Ransomware attacks have evolved to target critical infrastructure and manufacturing facilities. These attacks have led to large downtime windows for the target organizations. Regulatory and Compliance Frameworks: Governments and regulatory bodies are recognizing the need to establish cybersecurity standards for critical infrastructure sectors. Compliance requirements and guidelines are being developed to ensure the security of OT and ICS systems. Supply … Continued
Data is an ever-present and growing benchmark in all industries. However, sometimes it may feel like it’s cloaked in mystery. There’s often a question of, why is collecting process data important? You then start to go down the rabbit hole trying to answer…..How do you get data out of your equipment? How do you decide what to collect? After assessing the previous questions, how do you test, and if necessary, validate that all of that data is being collected as expected? Understanding a holistic approach to data collection and validation from an AVEVA PI Historian System can help provide a high-level, big picture understanding to the importance of process data. With more than 10 years’ experience as a group engineering manager, Matt Martin has a depth of experience with integrating AVEVA PI Systems within sites of various sizes – here’s what I’ve learned: Why is the Importance of Data Increasing? The “why” question is the easiest one to answer. Everyone can agree that in the era of modern manufacturing, extracting data from your manufacturing processes is critical to a business’s success and ability to grow. This includes every level of production and management: Provides the operator at the plant floor level the ability to adjust and better control the manufacturing process on the fly Allows engineers better insight into trends and behaviors of the overall process to improve designs from both a throughput and efficiency standpoint Produces KPIs (key performance indicators) for managers and executives to assist in their decision making Having large representative data sets is becoming even more important with the advent of AI and more specifically machine learning. Allowing these technologies access to your systems data helps identify patterns, overall system behavior, and correlations that were previously near impossible for the human eye to uncover. Now, How … Continued
Upgrading an Irrigation District Water Treatment Plant’s Control System
E Tech Group upgraded a decades’ old, primitive control system in an irrigation water district treatment plant to allow for remote monitoring and control of some of their systems, more reliability, and less maintenance.
The Project: Modernize an Antiquated Control System for a Water Treatment Plant
An irrigation district water treatment plant’s components were reaching end of life from their original installation E Tech Group executed almost thirty-years prior. The relationship built and maintained over those years, coupled with a quality system kept running with standard maintenance and a recent upgrade to their SCADA System computers and software allowed for E Tech Group to step right back in to retrofit and upgrade this facility’s control system. Our team worked from a Design-Build approach, working directly with the client to develop the scope of work and design:
The Scope: Drowning in Problematic End-of-Life Symptoms
The team was asked to tackle the upgrade of eleven control panels. The main plant’s PLC CPU was an obsolete Rockwell L61 that needed to be replaced with a current model, firmware and software. The main driver for this section was the fear of equipment failure and risk of extended plant downtime.
The raw water pump station and washwater panels used obsolete Rockwell L34 PLC CPUs which had recently begun to degrade to the point they began losing programming at random times and also required frequent reprogramming. This was a serious inconvenience to the operations staff as there was little to no predictability of these outages, which meant the problem could arise at any time, day or night.
The only current solution the client had found was to run manually for an extended period of time until the issue was temporarily resolved, which required an increase in maintenance labor and cost. The plant remote I/O panels were connected via ControlNet and the client wanted to move to EtherNet/IP providing a more stable and reliable connection. A need for redundancy in the main plant PLC and network was identified, leading to the placement of the remote I/O in a ring topology.
Wanting to implement more remote monitoring, the decision was made to add remote control and monitoring for the plant air compressor system as the existing was local control only. The last piece identified for this upgrade was to remove two obsolete physical PID controller devices from the main control panel and replace them with PLC software based PID controllers.
The Challenge: Maintain Uptime During a Control System Upgrade
Before we could begin executing design and installation with the defined scope of work, there was one large restriction that would require a creative solution to navigate. The plant would need to remain operational during the retrofit. The team would be allowed downtime for each panel, but not extended downtime for the entire plant.
To maximize efficiency our team modified the PLC program to allow them to maintain a hybrid system containing both the ControlNet and EtherNet/IP remote I/O modules. With this modification they were able to keep downtime to less than an hour per panel for the remote I/O conversions.
An environmental challenge that no one could have accounted for during planning was that a mouse had chewed through one of the fiber-optic links between control panels. This forced the temporary reuse of the existing Ethernet radios to keep the remote I/O rack online and functional until the fiber-optic link could be fixed.
The Solution: Define, Design & Install a New Wastewater Automation System
With the scope of work now defined, potential challenges identified and accounted for, our automation system integrators would move forward in the development and installation of a solution that would move this facility into the twenty-first century with increased reliability, decreased downtime and lower maintenance costs.
E Tech Group’s custom control system solution included:
Designed a new Ethernet and Fiber Optic network topology.
Developed a retrofit design package detailing the modifications that we were going to make to each panel.
Developed a retrofit commissioning plan detailing the process and sequence of how we intended to modify the system.
Migrated the PLC program from version 16 to 30, migrated the L34 CPU programs into the main PLC program, added the plant air and PID programming.
Platform – Upgrading the system with newer Rockwell components made the most sense as the programming could be migrated, and the existing I/O modules could be reused.
Replaced Rockwell L61 CPU with Rockwell L81 CPU and upgrade firmware/software – v16 to v30.
Replaced Rockwell ControlNet communication adapters with Rockwell Ethernet/IP communication adapters.
Replaced Rockwell ENBT Ethernet modules with Rockwell EN2TR Ethernet module.
Replaced Rockwell CNB module with Rockwell EN2TR Ethernet module.
Replaced Rockwell L34 CPU to EtherNet/IP remote I/O conversion – moved logic to the main plant PLC
Partnership – Installation of new Ethernet and Fiber Optic infrastructure
The client self-performed the following work under the guidance and direction of E Tech Group
Installed conduit needed for the new network infrastructure.
Installed Ethernet and Fiber Optic cabling between panels needed for the new network infrastructure.
Partnership – Commissioning
Operations worked closely with E Tech Group during commissioning to keep the plant running during each panel cutover.
The Result: Retrofit Increases Reliability & Grows Relationship
Our team was able to deliver the proposed solution on time and on budget, all while keeping the facility operational, aside from the agreed upon downtime for each panel. The plant is now running optimally using current automation products, along with an inventory of spare parts onsite should something happen, reducing the risk of long-term downtime significantly.
The system is more reliable now that the L34 PLCs have been removed; no more middle of the night surprises. Operations is now able to monitor and control the plant air system, eliminating the need for manual operation and consolidating resources to a more central location. The client was incredibly happy with the quality and proficiency of our work and now allows us to list them as a reference.
