Smart grid analytics: Why you should fire your coders (and hire solvers instead)
By: SGN Staff
Editor's note: JD Hammerly formerly ran North America for Alstom T&D (since purchased and renamed Areva Grid). He's one of that rare breed with a deep understanding of the technical side AND the financial aspects AND the business issues AND the sales side. This broad knowledge base makes him adept at spotting trends and their implications. Over lunch recently he told me his firm was getting amazing results from a transformational approach to coding aimed at data analytics. Since readers tell me analytics is simultaneously their biggest opportunity and their biggest headache, I asked him to share his approach with Smart Grid News. - Jesse Berst
By JD Hammerly
The electrical industry currently uses a myriad of applications for data analysis. These applications take in data, apply algorithms, and present results. But they are built "one-off" with a heavy emphasis on coding. Typically, 75% of the total code is dedicated to "infrastructure" â€“ data preparation, application flow, storage of results, etc. Thorough testing of the algorithms must await the completion of that extensive infrastructure code.
Historically, this approach was effective. But the smart grid has meant an enormous increase in the available data, which resides in a â€œsystem of recordâ€ archive scaling to trillions of records representing tens of petabytes. This data tsunami puts the traditional approach in jeopardy.
Results that are faster AND better AND less costly
The "solver" approach greatly speeds results. Since the developers no longer write traditional code, the bulk of their time is spent on a) formulating the problem and b) analyzing the results. Applications move quickly from concept to rapid prototyping, with software releases occurring every two to six weeks.
This approach also improves quality. Domain experts use the algorithms without any need for coding. And they do it in stages, with accuracy verified at each stage. This staged process lets them test alternatives with direct comparison of the analytical results. Lastly, dividing applications into data integration, algorithms, and presentation layers allows parallel development and parallel evolution.
The solver approach reduces initial and on-going costs for the developer and, potentially, for the end user as well. The suppliers of solvers and mathematical engines spread their costs across multiple industries. Further, they willingly support the latest hardware advances such as parallelization to improve overall performance.
Our knowledge-driven future
The coming transformation of the electric industry requires a shift away from obligation-to-serve electric companies towards knowledge-driven service providers. The analytic applications essential to unlocking that value cannot rely on architectures designed for small data volumes and the code-centric applications of the past.
The business value locked within the explosion of smart grid data will enable transformation of the electric industry. A new breed of application will address tomorrowâ€™s challenging issues by delivering accurate, reliable results at a lower cost of ownership. The future belongs to big analytic applications analyzing big data. And those applications will be built using a rapid-prototyping "solver" approach rather than yesterday's code-heavy methods.
JD Hammerly, CEO and co-founder of The Glarus Group, has extensive electric industry executive experience with Areva, Alstom, ESCA, Harris Corp. and multiple entrepreneurial ventures. With Robert Moisan, he recently co-founded Pegasus DMA, a solutions company, pioneering next generation analytics layered on a smart grid information repository managing trillions of records and supporting hundreds of users.
You might also be interested inâ€¦