Georgetown, Kentucky

Do it yourself SCADA Replacement

The story of Georgetown, Kentucky

By Christopher Little, Trihedral Engineering Limited

Shawn Derrington of Georgetown Municipal Water & Sewer Service offers advice on building the SCADA system you need

When the Georgetown Municipal Water & Sewer Service evaluated the cost of upgrading their existing SCADA software in 2006, they decided it was time to consider replacements.  After much research, it became clear that not only was it possible to get all the features they were missing; they could do most of the configuration in-house.

Named in honor of George Washington, Georgetown Kentucky is home to 20,000 residents, a private liberal arts college, and one of the largest car manufacturing plants in the US.

The Georgetown Municipal Water & Sewer Service (GMWSS) is comprised of a water plant, three waste water treatment plants, a few dozen lift stations, and five water towers.  These are monitored and controlled by approximately thirty remote telemetry units (RTU’s), each communicating with a central base station RTU over spread spectrum radio. 

Shawn Derrington is the IT Director and Microsoft Certified Systems Engineer at Georgetown Municipal Water & Sewer Service.  When he came aboard two and a half years ago he inherited a SCADA HMI software application that posed a variety of challenges for its users.  “There was nobody to maintain [the original software].  There were lots of things we couldn’t do with it or we didn’t like about it, but we had no way to make changes.” says Derrington. 

Challenges included:

1. Information could only be monitored via two unconnected HMI computers:  “The water plant operators had the main computer at the plant for reference and system control. If anyone else wanted any system information, they had to go there or call and ask. The second computer was at our Wastewater Plant #1. It monitored all of the pump stations for runtimes, power fails, seal fails, etc. It was only available in one building on the far side of the complex.”
2. Limited report manager:  “All we had in terms of historical data were these daily and monthly reports of runtimes and flows.  The reports were stored as individual text files so there was no easy way to compare or compile data.”
3. Third party alarm dialer:  The system used a third-party alarm dialer which posed serious compatibility issues during software upgrades or version replacements.

Also, GMWSS wanted to eventually tie in several RTU’s that were being monitored by a separate cellular-based SCADA system.  There was no practical way to implement this with their existing SCADA central.

Once it was decided that their existing SCADA HMI simply was not meeting the utility’s needs, they started looking at upgrading the system.

Priorities included:

1. Sharing real-time and historical data across the network with engineers, managers, and a variety of other users
2. Comprehensive reporting, trending, and handling of historical data
3. An integrated ‘all-in-one’ solution without version compatibility issues and expensive add-ons for mission critical features
4. The ability to make changes to the system in-house as the needs of the utility evolve
5. The ability to communicate with existing monitoring devices as well as devices from a variety of other manufacturers

A whole new SCADA central:
Derrington realized that not only would it be expensive to expand his existing system to include new features, they would have to continue paying yearly support fees for these new features indefinitely.  There was also the issue of making changes as the infrastructure continued to grow.  “I’ve got a computer background and so what we ultimately wanted to do was to make our own edits and change things around.  We started thinking that a whole new system was the way to go.”

GMWSS worked with local system integrator Rawdon Myers to develop their PLC SCADA program.  This laid-out the groundwork for the HMI requirements. With this information, they began the process of identifying possible SCADA software replacements.  After significant research, GMWSS compiled a shortlist of new SCADA contenders.  Some of these were eventually eliminated because they also required third party components such as alarm dialers and therefore presented the same issues of compatibility as their current system.

GMWSS eventually chose VTScada from Trihedral Engineering after receiving positive comments from Rawdon Myer and after visiting other existing VTScada installations in the area. 

HMI database conversion:
It was originally assumed that the HMI database for their existing SCADA central would have to be rebuilt from scratch; a job that would normally require a significant amount of time and money.  “We had originally planned to hire a system integrator to build the new system, while giving me basic training such that I could maintain the system going forward.” 

Instead, Derrington adopted SCADA software that included an integrated database conversion utility.  “That was a huge thing for us, to be able to convert our old database.  That saved an untold amount of time, and prevented data entry mistakes.”   Database conversion is an approach used by many utilities and system integrators alike for saving time and reducing the likelihood of costly errors.

“After doing some of the web tutorials, I soon realized that I could do most if not all of the development myself.”

Phase I: Planning

Derrington suggests that anyone considering this kind of project should take the time and plan as much as possible in the beginning.  “We had a pretty comprehensive idea of where we were going with this because we took our time and spent several months looking at different products and testing.”

It is important to take a detailed look at the facilities and determine what is actually necessary.  “We have a couple of lift stations that are very small and were simply deemed unnecessary expenditures for monitoring.  We spent a significant amount of time just documenting the system tags, radio settings, PLC logic, etc. for our entire system.”  Having this information in one place can pay off for years to come, especially when working with outside contractors or system integrators.

“Have an idea of your page layout and settle on it so you don’t have thirty pages that look different.  Set up a standard, so operators expect information in the same places.   Previously, the controls and pump information that our operators needed to see were spread across 3 different pages.” 

GMWSS used aerial photographs of Georgetown’s treatment plants and pump stations to create visual navigation that would be instantly familiar to operators and staff.  “Our city and county has an Aerial photography project so we were able to get that imagery from them.”   Another simple and effective way to instantly convey important context for information is to use standard sets of colors for groups of pages or tags. 

NOTE:  While it is important that applications look good to users, excessive use of graphics can actually distract from the information being displayed and drain system resources.  As Derrington points out, “The important thing is to allow [operators] to get on with what they are doing.  It’s just a tool for them.” 

Phase II: Development

“The first treatment plant took us a couple of months.  We bought the software in April of that year but we didn’t make it completely live until the middle of October, largely due to the documentation that needed to take place.

“We had probably three months where we had both systems side-by-side when we were testing and making sure the data was the same.”

By the time he began developing the new system; Derrington had already created several working SCADA applications based on step-by-step tutorials.  This experience honed skills that helped him work more quickly and efficiently.
 
Copying & pasting system pages and tags is one such example.  The key to making this technique work is to make new pages and tags as generic as possible.  “Even though there are twenty-five pump stations, probably 80% of each pump station is identical so I can just copy a page and change the tags. “

It is important not to underestimate the time commitment involved in doing this work in-house.  Although Derrington didn’t have to abandon all his normal day-to-day duties, “It did take a large part of my time.”  Many utilities can not dedicate these kinds of resources and are more comfortable hiring a system integrator.

 

Support:

Though he configured the application himself, Derrington stresses the importance of choosing a manufacturer that can provide the technical support needed to support that process.

“We like that it’s a smaller company in that you can actually talk to people that you know.  They are always easy to get a hold of or will get back to you quickly.  You never have a concern that if you leave a voice mail or an email, that it’s going into oblivion. “

The Final Result:

Through research, planning, and a little help from their friends, GMWSS now has a SCADA system that meets all their requirements and can grow with them.  “We are thrilled with what we are able to do in-house now and the data we can share among the departments is vastly superior to our prior system.”

The integrated internet server and alarm dialer allow mission critical information to be securely shared both inside and outside the organization.  Derrington points out, “With virtually no training, users can simply select the object or process that interests them and find out what it’s doing…without calling me!  If I’m off duty supervisors get a call, they can VPN into our network and see the status of certain things without having to drive in.”

Derrington stresses the importance of their new integrated trending utility. “Our operators can click on ANY tag and get a real time trend.  Another click and they can change the time scale.  They can add any other tags to that same trend simply by clicking its tag. This is incredibly valuable to us. We don’t have to anticipate every trending need they may have and create a separate page for it.  Also we don’t have to go through a report generation process to see trend information. “

Thanks to an extensive library of device drivers, the open SCADA system can incorporate a wide variety of new hardware.   “In the next few months we will bring five new lift stations and a third wastewater plant into our new system. They are currently monitored by a cellular-based SCADA system. We will use their OPC Server application to port that data directly into VTScada to make all telemetry data available in one place. That’s going to be our next project.”

Now that GMWSS is able to make changes to its own SCADA system, Derrington is enthusiastic about the future.  “I realize not everyone may enjoy [creating their own SCADA system], but I look forward to making changes and doing new development.  We couldn’t be any happier with how it’s turned out so far.”

Shawn Derrington is the IT Director and Microsoft Certified Systems Engineer at Georgetown Municipal Water & Sewer Service. 

Images reproduced with permission from the Georgetown Municipal Water & Sewer Service.