Managing huge data with smart meters


By Christine Easterfield, Principal Analyst, Cambashi

Christine Easterfield, Principal Analyst, Cambashi

There are many business drivers pushing utility companies to improve efficiency -- from coping with the changes in raw energy costs to government targets for reducing emissions. New technologies are arriving to address these new goals and targets, and it is one of those areas where the total is, potentially, much greater than the sum of its parts.

One technology showing more and more evidence is the smart meter, which have been an emerging technology for some time but are now a reality in many territories. The installation of smart meters drives a number of benefits on both sides due to the availability of regular, accurate readings. For example, for the consumer this means doing away with uncertainty of estimated bills, and for the utility it means better financial control and more accurate data on energy use, which in turn should result in improved demand management. Whatever the driver, the readings the meters provide have to be accommodated in the back office systems of the energy companies.

Think about what that means -- traditional meters are read by regulation once a year and read, or more probably estimated, each quarter in between. So that's four readings a year per meter for each of, say, 1 million meters for an average utility, around four million total. A smart meter can give an accurate reading every half hour -- that's 17,500 readings per meter or 17.5 billion readings per year for the average utility.

Not big data -- huge data

According to the EIA (the U.S. Energy Information Administration) there are 20 million smart electricity meters installed in the U.S., each providing a minimum of hourly readings. In Europe, ERDF plans 30 million smart meters in France alone. Not just big data -- huge data, surely. 

In practice of course, those potential meter readings are not necessarily all reported back to the utility -- a daily snapshot is a huge improvement on the data held about the current state of energy usage -- but it is still a considerable challenge to handle that level of increase in data traffic.

Why bother even managing data, if it's not put to good use? Utilities must concern themselves with what value can be gained from manipulating and analyzing the data. ___________________________

Communications networks, storage, disc speed all need review to ensure that the data collected is available in a timely fashion and kept secure. But why bother even managing the data, if it is not put to good use? Utilities must concern themselves with what value can be gained from manipulating and analyzing this data. Admittedly, the latest business intelligence tools no longer require carefully constructed data structures to be defined before any value can be gained, but analyzing the data for the sake of it is just job creation for your tame statistician.

Utilizing business intelligence

Business intelligence software now available to handle 'big data' makes a virtue of finding patterns in unstructured data. While your 'big data' as such is not necessarily unstructured, what it may not do is share a structure with other data sets in the organization. So the real benefit of collecting this data and analyzing it with the latest business intelligence tools may be in manipulating multiple, previously unrelated data sets. Analytics tools, which are becoming increasingly available, can combine meter readings with network telemetry readings and patterns can be found and offer insight into how customer behavior impacts network status.

We know, anecdotally, that the increase in the use of air conditioning in the warmer climates puts a strain on energy supply that can result in catastrophic failure of the network. What if data on the rise in energy demand could be combined with short-term weather predictions for increasing temperatures and equipment readings showing the approach of critical limits? That might mean the ability to predict potentially catastrophic failures and the opportunity to manage a way out of the resulting blackout. Simulation tools could use the data to model where failure might occur --enabling the utility to establish the boundaries and limits of their operating parameters.

So the initial driver for deploying smart meters -- improved billing; better demand management -- delivers the efficiency planned, but other benefits accrue if additional tools are deployed to get the most out of the resulting data. The crucial factor is to identify what benefits could be achieved through close analysis of the data -- and not simply to crunch the numbers for the sake of crunching.

About the Author
Easterfield is a Principal Analyst with Cambashi. Her experience has been in geospatial asset management for the utility industry, assessing market needs and opportunities, managing customer requirements, liaising with development teams and running global product introduction programs.