Build the data infrastructure and analysis software able to collect, store, process and visualize the massive data produced by the new generation of ultra-high-resolution power grid monitoring devices.


Public utilities who are in charge of monitoring and controling the electrical grid to ensure its stability and reliability, traditionally do so using so-called SCADA information systems (Supervisory Control And Data Acquisition). At stake is the ability of the utility to detect problems as quickly as possible, and with as much detail as possible—a most dramatic failure example being the Northeast American blackout of 2003 that affected 55 million people during two days.

Wide Area Monitoring Systems (WAMS) are the next generation of energy transmission information systems. They provide an enhanced situational awareness of the state of the electrical grid, with high time and spatial resolutions, offering identification and analysis of dynamics not observed by conventional SCADA systems.

The latest monitoring systems provide an enhanced situational awareness of the state of the electrical grid, with high time and space resolutions

WAMS relies on a large number of GPS-located Phasor Measurement Units (PMUs), a type of device which measures, at a very high frequency, the characteristics of waveforms from AC electric signals: magnitude and phase of voltages and currents are collected with up to 48 samples per cycle for a 50/60Hz current.

By the end of 2016, 30 PMUs will be running across Europe, and GE aims at quickly scaling its capability to 1000+ systems world-wide

The large volumes of these geolocated data are processed by Phasor Data Concentrators (PDC) which synchronize the sampling with 1-microsecond accuracy and transmit to the WAMS at rates up to 60 samples per second. They are the intermediary step that reduces the data-processing time and computational requirements needed to time align, translate, error check, and/or change the data rate of data from multiple PMUs.

Use case

GE has developed a first Predix application for WAMS that currently processes the data of 3 PMUs. By the end of 2016 there will be 30 of them installed across Europe. GE aims at quickly scaling this pilot software so that it can handle 1000+ PMU systems.

Challenge objectives

The primary objective of this challenge is to develop a cloud architecture and analysis algorithms in Predix which can ingest, store and process for quick-response analysis, the data generated by a large WAMS system, comprising more than 1000 PMU devices.

Five directions of possible work are identified:

Optimal storage strategy

The challenge is to make the most efficient use of storage for fast system performance, but without excessive storage costs incurred. Decisions on how to assign data to time series or BLOB store, and how to reference and retrieve data effectively are critical to high performance with the large data volumes produced by WAMS systems.

Previously, summary/derived data has been evaluated on real-time data and stored, but there may be other possible approaches including re-calculating on request. Calculating the performance data on incoming data is not flexible if new or revised processing is required.

Machine-learning pattern detection

When the power system is stressed, there are changes in behaviour of the power system, such as:
• phase angles between measurement points increase
• the background noise level increases
• likelihood of occurrence of disturbances (significant changes in values and “ringing”)
• likelihood of multiple disturbances occurring close together.

Also, there can be patterns between non-synchrophasor data, such as windspeed, import/export, estimated inertia that may be of interest for analysis together with the WAMS data. There would be a value in drawing relationships and predictions between measured data from multiple sources. Estimates of values looking ahead, such as weather prediction and day-ahead market estimates may be of value to predict system stability and responses.

There may also be a value in using a system model for training a machine learning approach to manage system conditions and event sequences that very rarely occur in real power systems.

A methodology and toolbox for identifying relationships between many variables, enabling long-term pattern matching and selecting/analysing interesting periods of data would be of value.

Data cleansing

All big data analysis must deal with bad data. This may be marked in the incoming data as poor quality, but may also be identifiable only as inconsistency over time (e.g. outlier and out-of-range samples), inconsistency between locations, relationships between signals (e.g. a low voltage magnitude can be valid while the associate angle and frequency will be invalid). Furthermore, data errors due to GPS blocking or spoofing can be particularly challenging.

The data cleansing challenge involves a mechanism to identify, flag and/or correct bad data, and avoid it distorting results.

Access rights

Some WAMS data is non-confidential and can be made freely available to any subscribing entity. For example, frequency can be measured from any wall socket, and contains no sensitive information that could be used to interfere with the energy market.

Other data, normally related to current in circuits, could be used to obtain sensitive competitive information, such as a generator’s bidding strategy.

Some data series may be accessible to multiple entities, but not to all users. For example, power from a battery storage operator’s sites may be available to:
• The battery storage operator’s own users
• The relevant distribution network to which the device is connected
• The relevant transmission network to which the device is connected
• The transmission system operator (which may not be the same as transmission owner).

Researchers and analysts may also have access to data, but with some restriction, for example, not the most recent 1 hour of data, or with unidentified circuit names.


Beyond mere data ingestion and storage, the demonstration could be furthered by developing a GUI enabling minimum exploration and analysis of the WAMS data:
• An overview geographical dashboard – with live alarm status of the entire WAMS
• Charts and Graphs — for both live and historical data, including strip charts, locus plots, histograms, polar charts, etc.
• Alarms and Events — these are clearly shown on map/topology views and in event lists


Live data are already being collected from the three PMUs in place, and are available for prototyping and analysis. Each data point, collected and transmitted at a 50 to 60 Hz frequency, currently contains:

  • the frequency
  • one voltage phasor (a complex number in polar format)
  • a timestamp.

Ultimately, PMUs are expected to measure three-phase data, and it will be common for them to send 6x three-phase phasors, on top of the frequency and frequency time-derivative. All data is stored for one year, and detected events for 5 years.

In the current challenge context, the 1000+ PMU situation will need to be simulated from the existing.

On top of the raw 50/60Hz synchrophasor measurements data, a WAMS system deals with other types of data:

  • Raw data at 200/240Hz at a subset of measurement points
  • Derived data such as oscillation analysis results, at typically lower data rate e.g. 5s updates, but with more elements in a series.
  • Other derived summary data like downsampled versions of the raw synchrophasor data, e.g. 1-sec updates of min/max/average, or summary values of performance, e.g. slow-scan measure of the variations of values over periods between about 1-sec and 15-minutes.
Partner team

This challenge is sponsored by GE Digital.

Robert Plana GE Digital Foundry Europe Coordination

Robert Plana

GE Digital Foundry Europe


Douglas Wilson Chief Scientist GE Grid, Psymetrix Operations & Solution expertise

Douglas Wilson

Chief Scientist GE Grid, Psymetrix

Operations & Solution expertise

Business scenario

The core data storage & processing module can be deployed on Predix as a service accessible to third parties for a price following the platform’s pricing principles (

A more advanced app, providing a graphical user interface and direct business-value insights, could become one of GE’s offerings to its clients, in which case a licensing agreement would be concluded between the app developer and GE.