The meter data management pitfall utilities are overlooking
By: SGN Staff
By Jesse Berst
I was interviewing experts for an upcoming webinar on meter data management (MDM) when I first heard about a smart grid pitfall I'll call "forgetting the foundation."
When they tour mansions on TV, they never zoom in for a close-up of the foundation. They spend all their time on the rooms and the decor. Yet if you perch a giant mansion on a foundation intended for a small two-bedroom, the whole edifice will ultimately topple.
The same with meter data management. If you perch an MDM system on a database foundation intended for an ordinary business application, the whole project may tumble.
That's the painful lesson some pioneering utilities have learned the hard way. Database technologies that are perfectly suited for run-of-the-mill business applications often can't keep up with the flood of smart meter data. Sadly, a few unlucky utilities have been "blindsided," warns Dr. David Shipman of IBM's Energy & Utilities practice, who has 20 years of experience in information technology for utilities. "Things worked fine at pilot scale, but broke down when they went to deployment scale."
Because they didn't have the right foundation, the utilities had to pull out the original database and swap in another more suited for the task.
You won't be surprised at my advice: The better strategy is to get the foundation right before you build your MDM system on top.
Choosing the right foundation
Let's say you move your meter reads from once every month to once every 15 minutes. You suddenly have 3,000 times more data to cope with. And that can wreak havoc with data acquisition, data analysis and data storage.
Data acquisition issues. "Traditional" databases use a SQL model. They post each meter read as a transaction, one at a time. But that method can't suck in data fast enough when you pump up the volume 3,000 times.
Data analysis issues. You are not collecting all this data for the fun of it. You want to use it for things such as:
Â· Detailed load forecasting and analysis
Â· Customer segmentation for demand response programs
Â· Etc., etc., etc.
But you won't be doing any of these things if data mining and analysis is too slow. In extreme cases, "We've encountered queries that took as long as 18 hours to run," explains Shipman.
Data storage issues. It's easy to overlook the costs of storage. But remember... we're talking about 3,000 times the data. "A database with world-class compression can reduce storage requirements by 50% or more," explains Richard Wozniak, an information management program director for IBM Software Group and another 20-year veteran. "For a large project, that can easily means savings of $100,000 per year or more just in storage costs."
And let's not forget regulatory compliance and retention requirements. Most utilities don't have the option of just throwing away big chunks of data because they're choking the database.
Checklist for success
I'll give you a quick summary of the things to look for when you're architecting your own foundation. For more detail, join me for a meter data management webinar on Wednesday, June 08, at 1:30 p.m. Eastern (10:30 a.m. Pacific). I've recruited David and Richard to coach us and give us their hard-earned lessons from the front lines. The webinar is free to SGN readers while space remains.
Here are five things to consider when choosing the foundational database for your meter data management system:
Â· A special time series data type to help with the data acquisition and data analysis issues
Â· World-class compression to help with the data storage issue
Â· Low overhead and administration
Â· An ecosystem of add-on tools and applications
Â· Field-proven products (look for products that have been used for things such as airline reservations and financial services, where the volumes are even greater and the financial stakes even higher)
Have you encountered these or other issues when implementing meter data management? Use the comment form below to share your experiences. And click the link to register for the meter data management webinar.
Jesse Berst is the founder and chief analyst of Smart Grid News.com. He consults to smart grid companies seeking market entry advice and M&A advisory. A frequent keynoter at industry events in the US and abroad, he also serves on the Advisory Council of Pacific Northwest National Laboratory's Energy & Environment directorate.
You might also be interested in ...