It’s 5:00 p.m. on a Friday. You’ve worked hard all week, and you’re ready to relax and unwind. You have plans — anything from having dinner with your family that has been planned for a week to heading to your kid’s sporting event to working out at the gym or just leaving town on vacation — whatever the plan, you’re ready to roll. But – at 5:01 p.m., your phone rings. It’s your on-site operator. The news isn’t good. It’s the end of a batch run and a control valve won’t close when it should, causing extra water to enter a tank. Your team has tried to resolve the issue but the problem persists. It’s affecting production and presents a safety issue, requiring immediate attention. What are you going to do? Stop what you’re doing and head back into work.
Welcome to the “5:01 Club” – aptly named for that select group of people whose plans are disrupted after normal work hours. As you know, stuff happens; it’s inevitable. Depending on how your manufacturing facility operates, problems can and do arise. Those ultimately responsible for operations must “FIX IT.” How long it takes you and your team to respond is critical to the safety and overall well-being of your staff and manufacturing facility.
Oftentimes, your in-house team can handle the various day-to-day automation system and process situations that come up. However, as described in the above scenario, problems often occur during the evening, holiday, vacation or weekend hours. In some cases, you have little to no staff support or your team simply does not have the bandwidth or level of experience to effectively diagnose system failures or other process issues. In these instances, you may need additional support for the short term. This is where a third-party remote management global operations center can be an effective alternative.
PlantFloor24® Responds
Let’s look at a real-world remote management global operations center: MAVERICK Technologies’ PlantFloor24®. According to Ram Ramamoorthy, operations manager, the PlantFloor24® team uses industry-leading best practices to provide 24 / 7 / 365 application support, remote management services andproduction optimization solutions. “Our team has the unique ability to develop applications, test products and run a global operations center,” he said. “They must first determine if an issue is an instrumentation, operator training, programmable logic controller / human-machine interface (PLC/HMI), manufacturing execution system (MES), third-party solution or an enterprise resource planning (ERP) issue. The team can then monitor the connected enterprise system and proactively resolve issues.”
To see the PlantFloor24® team in action, let’s revisit our batch manufacturing scenario. This time, however, the answer to the question, “What are you going to do?” is different and immediate. In this modified scenario, your on-site operator calls you to report that the valve won’t close when it should and that the team has tried to resolve the issue but the problem persists. The team is contacting the facility’s third-party remote management global operations center: MAVERICK’s PlantFloor24® team. Now, how does this scenario play out?
The real-world dialogue begins with Evan Pederson, senior engineer at PlantFloor24®, answering the incoming call from the operator:
PlantFloor24®, this is Evan, how can I help you?
Operator: Hi Evan, we have a situation with a control valve not closing when it should, causing extra water to enter the tank. We are not able to figure out why it didn’t close automatically at the end of the batch.
Evan: Is the team safe, and are you able to manually shut off the control valve?
Operator: Yes, the team is OK and the valve has been closed manually.
Evan: Good to hear.
[Evan then asks a series of clarifying questions to assess the situation further and to get an idea from the operator of what has already been done.]
Evan: Thanks for the additional information. We will diagnose your system on our end and get back to you immediately. What is the best callback number to reach you?
[The Operator provides the number and thanks Evan for the help. The call ends.]
At this point, the PlantFloor24® team of technical control system analysts immediately connect to the facility’s system using dedicated virtual machines for security purposes. They perform an initial assessment of the facility’s system code and quickly find the logic associated with the malfunctioning valve. Evan calls the operator back with the team’s findings.
What was the problem and outcome?
The logic controller was not reading the position of a level switch correctly. The operator confirmed that a new switch had just been installed recently. In checking the switch, it was determined that the on-site electrician had not wired it properly. Once the switch wiring was fixed, the problem was solved. Soon after the project was completed, the PlantFloor24® team sent an issue summary report to the operator.
Hopefully, the above scenario is not a common occurrence in facilities, but sometimes you just need additional help for preventative routine maintenance checks or for those incidences that need immediate attention. And when you do need help, you need expert help reliably and quickly.
Now, you may be wondering, “What is life like in a global operations center where they work 24 / 7 / 365?” and “What types of services are provided by a third-party remote management team?”
Life in a Remote Global Operations Center
To answer these questions, let’s hear from some of the staff at PlantFloor24®, who perform the remote management and monitoring support. According to Trevor Short, control system analyst for PlantFloor24®, life inside the global operations center is more than just answering the phone: “We are a closely-knit group of control system specialists who — no matter how hard the task — work together to figure it out,” states Trevor. “We all are very knowledgeable on various platforms and are very flexible when it comes to our schedules. Typical shifts run from working 12 weeks on days to 12 weeks on nights, and then the schedule rotates.”
Based on the various contracts the team has with clients, Trevor points out that they offer sustaining services, ranging from running weekly to monthly system checks to providing 24-hour
system support. Jordan Clark, senior control system analyst, explains: “We provide customer support. For some clients, we do weekly checks where we remotely login to their system and monitor their PLC controllers or their control system log files; we make sure there are no symptoms of a problem or any errors that are going to cause them trouble. We also monitor various types of equipment, including Windows servers, desktops, thin clients, motors, loops, analog valves, switches, controllers, etc. The team also performs development services, including HMI configuration/screen development type work, MES and ERP support.”
Working out of the global operations center, the PlantFloor24® team truly cares about the safety of the client’s personnel and ensuring systems stay up and running. “These guys would bend over backward to help the guy next to them and serve our customer,” says Trevor Short. “Depending on the severity of a project,” Trevor states, “the team would either work hand in hand with a client’s team, or they would reach out to their own project lead for added assistance.” Evan Pederson concurs: “We’ll bring in one of our escalation teams, senior engineers throughout the company who are assigned to the client, when needed.” This unique approach brings subject matter experts and technology leaders together, and that team remains on-call to the customer helping operations run smoothly at a fraction of the cost of full-time personnel.
In more urgent situations, the PlantFloor24® team has had upwards of 45 expert staff members working remotely in the field and on site helping to resolve a problem within multiple manufacturing facilities. “Most of the work is done out of the global operations center in Columbia, Illinois, but we support commissioning and other efforts on site as well,” explains Evan. “We have a very flexible approach to how we do the support. What we do for each client is customized. So, it’s not like a client signs up for support and [we simply tell them] this is what you get. For some sites, we have remote access [to the site]; some sites we don’t. Some we use our own ticketing system, and for others, we use their ticketing system.”
This flexible, customized approach paid off for one food and beverage customer. “The team successfully worked with the food and beverage manufacturer’s engineers and OEM vendors to develop a connected enterprise solution involving their control system, OEM packages and ERP solution,” said Ram Ramamoorthy. “The team was tasked with troubleshooting and solving instrumentation, automation (PLC and HMI) and MES issues. They provided support for a seamless transition to a new automated production environment while minimizing down time, improving safety, quality, and throughput when rolling out the automated solution concurrently across six sites. Besides troubleshooting and transition support, the team also provided pre-deployment testing support for application and training.”
As you can see, life in a remote management global operations center is never dull and brings its own rewards when a job well done successfully impacts and improves a client’s production processes and bottom-line. In the end, there are several remote management benefits and advantages to consider.
Remote Management Benefits
Engaging an external automation solutions provider with a qualified remote management team that provides automation services on a 24 / 7 / 365 basis can save in both the short and long term on the cost of downtime, as well as the cost to hire new personnel to handle various situations that occur.
As mentioned previously, your in-house staff can probably handle the day-to-day problems if they have the right level of experience and enough bandwidth. In many instances, however, time is of the essence, especially where staff might be at risk. Can you really afford to wait hours or days for them to resolve the problem?
Equipment and other process failures seldom occur without warning, but these warnings are often subtle, requiring careful analysis of data by an experienced team for early detection. Key indicators of impending failures are ignored if these analyses are not performed or are delayed. The result is often followed by downtime and unscheduled maintenance. Whatever the reason, delayed response to incidents can cause plants to
lose valuable production time and, in some instances, lead to damaged equipment or even personnel injury.
In these instances, remote management support can provide resources to develop and test required system or process changes, and work with your facility’s staff to remotely implement any necessary changes. In this way, it can improve your operations and reduce total cost of ownership.
Other benefits you can expect from a remote management team include the ability to:
- Provide immediate 24 x 7 response and proactive support
- Remotely identify, diagnose and debug automation equipment and process problems before they cause interruptions or reductions in production
- Identify and monitor the critical parameters that typically indicate most failures. Examples include hard disk usage, percentage of utilized memory, warnings generated by the automation system, network data andinformation related to equipment operation such as vibration parameters, temperature readings, etc.
- Monitor these parameters remotely on a 24 / 7 basis, using sophisticated statistical data analysis techniques when necessary, along with expert judgment
- Recommend proactive maintenance, avoid breakdowns and consequent downtime
- Provide continuous improvement services to processes and equipment
- Perform weekly or monthly system health checks
- Make remote corrections when abnormal conditions occur and / or notify facility staff to take action
- Perform system backup and recovery
- Run plant security assessments and remediation
- Perform inventory tracking and control
- Provide alarm response tied to KPI and IT monitoring plus response
- And more
A dedicated remote management team can be an effective means for dealing with incidents, helping to resolve problems quickly as situations arise. A global operations center, like PlantFloor24®, can provide a wide range of remote management services, tailoring each support contract to your specific needs while tapping several automation experts to directly support you and your facility either remotely or on site, if needed.
Don’t wait to find yourself a member of the “5:01 Club.” Consider using a remote management service to provide exactly the support you need, exactly when you need it. MAVERICK’s PlantFloor24® team is ready and on call 24 / 7 / 365.
