1

By Jeff Taft, Chief Smart Grid Architect, Cisco Systems

By now, you've heard how utilities are being swamped by the quantity of smart grid data. Here's what you're not hearing – there is an even harder problem on the horizon… namely, smart grid data quality. As Chief Smart Grid Architect at Cisco, Jeff Taft is in a good place to see what happens when you take the smart grid from pilot phase to scale. He stopped by not long ago to explain how hidden couplings could sabotage the smart grid. When he told me about the looming problem with smart grid data quality, I asked him to contribute again. Here's the first of a two-part series on smart grid data quality issues. – Jesse Berst

 

In the process of working on many smart grid pilots and deployments, I've discovered two data management issues that often trip up unsuspecting utilities. The first, described below, is the issue of smart grid metadata. The second, which I will cover in a separate article, is the challenge of ensuring the quality of real-time data.

 

The five classes of smart grid data

I think in terms of five different types of smart grid data, each with its own special challenges:

1.     Advanced metering data

2.     Waveform oscillography (e.g. samples of current and voltage waveforms)

3.     Telemetry (e.g. SCADA data)

4.     Asynchronous event messages (the real-time data I warned about earlier, e.g. smart device event alarms and alerts, plus command and control instructions)

5.     Grid metadata (data about the data that provides context, such as network models and topology that show what is connected to what in what order)

 

The importance of metadata

Since grid metadata provides the context for the other four categories, its accuracy affects everything else. And that brings us to the problem. Grid metadata is famously inaccurate. As in 20% to 50% inaccurate.

 

Yeah, that's right. I just said that grid metadata affects everything else. And that up to 50% of it is wrong. Think of a map. Even if the rest of the data is 100% up-to-date – the place names, the mileages, etc. – the map won't be of much value if 50% of the roadways are shown as they were in 1957.

 

We can fix some of the metadata problems through technology. But much of the solution depends on people. (See, I told you it was scary.)

 

A smart grid data quality example

Many of today's substations have two parallel data collection systems 1) traditional SCADA and 2) a substation gateway that supports various “smart” systems

 

So let's say that an input channel fails. The substation technician changes the input on the station RTU and informs the SCADA technician in the control room. But he forgets (or never knew in the first place) that the smart systems need to know as well. As a result, the SCADA system is fine, but our supposedly smart systems are now reading from a dead channel.

 

The biggest smart grid data quality issue

If I had to characterize the biggest metadata challenge, I would say "as built versus as operated." I gave an example just above. The substation was built one way, then changed and operated another. With the smart grid bringing constant additions and upgrades to all parts of the system, the situation has become very fluid and dynamic. Not only that, but with dynamic feeder circuit switching, topology changes can occur quite quickly under either normal or stress conditions on the grid.

 

With things changing every day, how do you make sure that every part of the system gets the memo? In a phrase: people and processes.

 

So let me confess. As an engineer, I believe in technology and I love to automate systems to prevent operator error. But that is not always possible. And that leaves us relying on business process engineering to make sure that human errors don't bring down the system.

 

Two tips for improving smart grid data quality

Here are two quick tips for improving smart grid data quality:

 

1. Use installation to do a reset. The rollout phase is a great time to clean up the metadata. Technicians are on site anyway. Ask them to verify the accuracy of the field data.

 

2. Create mechanisms and processes. Don't expect people to remember all the intricacies. Instead, build mechanisms to get the meta-data once, and then distribute it to all the systems that need it. (Hint: sometimes the hardest part is figuring out all the systems that are directly or indirectly dependent. Smart grid data is far more interwoven and interconnected then we realize at first.)

 

In summary, before you depart on a long smart grid journey, make sure your map – your metadata – is up to date. And that you have processes to make sure it stays that way.

 

Next time: The special challenges of integrating real-time data and ensuring its quality.

 

You may also be interested in ...

Next-generation asset analytics (combining operations data with back office data to monitor and manage field assets)

Next-generation asset management webinar (replay with downloadable slides)

Smart grid data management Q&A series (experts answer your data management questions)

Talk Back to the Author

Current Comments (7)

Leave a Comment

---

data quality

Jeff is absolutely correct and I would like to add another factor - scope of the data. As the Smart Grid is pushed to show some return, applications are necessary - e.g. demand response, theft. In his note, he identifes the scope as the substations, meters and (by inference) the distrubution grid which is correct for many applications. These three areas are, today, rarely cosidered as one project thus the information systems do not have the right scope to create these applications safely. As to the sychronization of information between SCADA and subervisory systems, this is truly an isssue. It can be solved by technology (auto update) but seldom is. The same problem exists with the update of the meta data as well.

pat kennedy - 04/27/2011 - 06:36

time synchronization

In addition to Jeff's and Pat's comments. We face much larger latencies than anticipated and are also avoiding the issue of edge processing. Particularily for PMU alarming systems that can be done directly at the substation interface.

Chuck Wells - 04/27/2011 - 07:10

The Training and Communication Solution

“But much of the solution depends on people.” Mr. Taft has that so right. And to finish that thought, those people depend on being trained and kept up to date. As an instructional designer and writer, I have fought this battle my whole career. As deadlines slip, all resources go to “just make it work.” But if “people” don’t know how to use it, the processes still don’t “work,” regardless of how brilliant the coding may be. If time and budget aren’t set aside to get documentation, training, and communication right from the start, costs that did not initially appear as budget line items sky rocket later.

John Morley - 04/27/2011 - 08:46

Robustness and Learning

Other complementary things we could do are 1. Create a highly decentralized, grid control architecture, consisting of a flexible hierachy of semi-autonomous agents, for the grid operation (and monitoring) system. Design that architecture up front to be able to roll with the punches of bad or incomplete information, in a manner analogous to how the Internet routes around damage and is somewhat robust to misinformation in ITS metadata. 2. Build in a learning system that can theorize about such errors in metadata, can discover inconsistencies in readings (compared to models, history, and expectations) and notify about them with plausible explanations.

Eric Hawthorne - 04/27/2011 - 15:11

Entelios

Jeff, our system is built exatly that way, as you recommended in your points 1. & 2. While we do B2B only and leave residential smart grid participants to our competitors, we are facing a shift in complexity from smart grid operation related problems to the issues while implementing SCADA interfaces.

Stephan Lindner - 04/28/2011 - 02:31

Smart Grid issues

In addition to the issues Jeff has mentioned, technical manpower required for analysis of data is very difficult to get. This causes the data, to become obsolete and redundant. This is based on experience from India.

hemant diddee - 04/28/2011 - 04:06

integrating the metadata from the entire HAN/P1901

"Asynchronous event messages (the real-time data I warned about earlier, e.g. smart device event alarms and alerts, plus command and control instructions) " also include many derived from the other data, such as potential electrical fire warnings based on the detected waveforms in the internal wiring. So not only the devices but the connections/circuits between require explicit modelling in the overall metadata, especially for fire/noise/brownout/outage isolation. Not thinking through the smart home, smart device and other home-area network (HAN) considerations before settling on a metadata schema is such a serious mistake I doubt utilities will survive it. The input of vendors like Verizon, who vend both the communications services and are rolling out metering and data management (and will likely do this on general-purpose gateways not crippled "smart meters") is needed up front, immediately. It's not power companies that know what this looks like. It's companies like Cisco, Verizon and those few utilities that operate as both communications and energy companies. They use different technology for this today (separate fibre and coax networks, some wireless, and a dumb service cable) than they will tomorrow (fibre only to the pole and transformer, therafter gigabit IEEE P1901 powerline networks to every outlet for both metering and all communicatinos). But the modelling will be the same. Other IEEE standards such as 802.21 deal with the handoff between wireless devices of all kinds, and handling various wired backbones is any easy add. Lots to say on this, but there has to be a forum for these conversations devoted specifically and only to that. A wiki, preferably, as W3 uses, not some kind of top-down PowerPoint-based lecture/lie. This is not an easy problem to solve. It'll take a decade to work the bugs out, and at every stage the specific problems/bugs/annoyances/malfunctions of a more complex integrated system will be cited as a reason to stop and go back to something stupider. However, at every stage the gains will justify it: fire safety, home security, vulnerable persons' medical monitoring, radical gains in demand side energy use reduction and peak shifting, reduced need to invest in new supply or expensive wholesale power, and of course a gigabit to every AC outlet... All of which is worth a few hundred bucks a month, but if delivered uniformly could cost less than one hundred per household, offsetting needed per-kw rate increases.

Craig Hubley - 04/28/2011 - 16:24

Leave a Comment

New to this Blog? Need some help?

Email: [?] We will send you an email with an appproval link you must click for your comment to appear.
Full Real Name:	[?] Real comments need a real name.
Subject:	[?] Please provide a short subject for your comment.
Comment:[?] Do not post commercial/sales messages here. They will be deleted. Try: pr@smartgridnews.com PLEASE NOTE:  All HTML codes will be removed from your comment.
CAPTCHA Security Code The numbers (0) and (1) are not used. [?] Please type the characters you see into the box above.
Remember my name? (Uncheck to forget.)
You can not edit or delete your comment once it is posted. Please check for spelling now. Your IP Address is: 50.17.171.254