Transforming Electric Utility's Performance with Data Analytics

Data Analytics and Utilities

People disagree on many topics in today's world, but most would admit that data analytics, as a powerful tool, is transforming many businesses in various ways. From marketing to sales, from operations to R&D, data analytics and its derivatives play an increasingly important role in almost every aspect of a business's ecosystem, with no exception to the power utility industry. The good news is, the utility, or power industry in general, is never short of data. On the transmission side, traditional data sources such as SCADA record basic electric parameters such as voltage, frequency, and power flow every 2 or 4 seconds for just about every bus and line in the grid. The phasor measurement unit, or PMU, measures the same quantities with up to 120 samples per second to capture the transient events in the power grid. On the distribution side, smart meters and sensors record data at the feeder or individual household level every few seconds. For deregulated states in the U.S., the Independent System Operators (ISOs) calculate and publish prices, generator dispatches, and load and renewable forecasts on a 5 or 15 minutes basis. The list goes on...

While utilities are sitting on ample opportunities to benefit from data analytics, the reality is that few have appreciated the data enough nor utilized them to the full potential to guide the decision-making process in daily operations compared to other industries such as finance and marketing. So how will the data help utilities transform their business? The answer is in many ways, and the following are just a few of many possibilities:

• Profit improvement: Advanced data analytics is a powerful tool to identify and execute profitable trading strategies for utilities that participate in an energy market. Data-based optimizations tend to improve the quality of energy bids and resource schedules, giving utilities great potential to unlock additional profits with bounded risk exposure in the energy market. On the operations side, data analytics can improve maintenance scheduling in the planning stage and even has the potential to predict the future grid conditions in near real-time with risk-based mitigation strategies.
• Cost reduction: Operating expenses account for approximately 88% of a utility's operating revenue 1 and increase yearly. Data analytics can effectively reduce human bias and assist the management to allocate their capital more efficiently by prioritizing high-value procurement. On the business side, people (data) analytics could reveal potential functional realignment opportunities, allowing management to maximize the value of human resources and improve employee satisfaction. In addition, with climate change and the increasing risk of wildfires, data analytics can help utilities evaluate the various geographic regions and develop tailored mitigation plans for each location based on its risk level.
• Reliability enhancement: Delivering electricity reliably is always one of the key metrics for a utility. Data analytics can increase the grid's reliability by predicting the equipment's failure rate and allow maintenance plans to be developed in advance to improve outage planning and minimize the impact of critical contingencies. Additionally, data analytics can better quantify the uncertainty of power flow, load level, and renewable outputs through predictive analysis, providing system operators and energy traders better situational awareness.
• Customer acquisition and engagement: Due to the shift toward a distributed grid, tomorrow's utilities are expected to face increasing competition, not from other utilities but emerging energy providers such as Community Choice Aggregation (CCAs) and Tesla-like companies. It means to traditional utilities that the advantage of a natural monopoly begins to fade, and customer acquisition and engagement will become key. Utilities should utilize data recorded by smart meters to recognize the (aggregated) user's energy consumption pattern and offer more targeted energy products to help customers optimize their energy usage.

Type of Data Analytics

The famous Gartner analytic ascendancy model below categorizes analytics into four types: descriptive, diagnostic, predictive, and prescriptive. Descriptive and diagnostic analytics analyze historical data and try to answer what happened and why it happened, while predictive and prescriptive analytics focus on the future and predict what will happen and what actions to take. The analysis will arguably become more complex as moving up the arrow but will also add more value and generates greater insights. This is not to say that one analytics is clearly better than the other. In fact, the need for the type of analytics varies at different stages of a company. In recent years, however, the trend is moving towards predictive and prescriptive analytics types 2.

Data Analytics at SMUD Grid Operations

Shortly after SMUD joined the CAISO Energy Imbalance Market (EIM) in April 2019, we decided to pursue a data-driven approach and initiated an analytics project to facilitate the after-the-fact EIM analysis, answering what happened and why it happened. Later, we added a predictive component to provide operators and traders the situational awareness of daily reliability and market operations. The project involves four steps: pulling data from the California ISO's Open Access Same-time Information System (OASIS) and Customer Market Results Interface (CMRI) portals and stored in the local database, cleaning and reconstructing the data, perform statistical analysis, and display to the users as shown below. The analytics part of the project was developed in R using various packages such as flexdashboard and shiny. The final product - an interactive dashboard - is hosted in Docker.

As illustrated below, the dashboard has five tabs, each displaying daily key EIM performance metrics such as real-time prices, congestion, intertie power flows, and generation dispatches. The user can pick a day from the date panel on the left and navigate through the tabs to review the metrics for that day. These displays focus on answering what happened and why it happened, which are critical in the after-the-fact analysis to learn from the mistakes and work toward operational excellence.

As demonstrated below, SMUD's EIM performance is also summarized on a rolling 7-day and 30-day (not shown) basis for engineers and management to track and take appropriate actions as needed without delay.

Recently, a predictive component is added to quantify risks and help engineers and traders understand how prices and flex requirements vary across different hours of a day. Based on statistical analysis using historical data, users will be able to predict the next day's price or flex requirement with a level of confidence and risk allowance. With this information in mind, schedules can be developed more accurately for real-time operations. Are these the most advanced data analytics techniques? No. The techniques used here are no more than data reconstruction and basic regressions. However, don't overlook simple techniques like these because the cost to implement something like this is usually relatively low, yet the benefit, which is what you should focus on, could be significant. 

We also have explored more advanced machine learning techniques to predict flows on a transmission path (see below) and the reserve requirements. Based on two weeks of testing data, the Long Short Term Memory (LSTM) model predicted the flow 25% more accurately than the current method on the power flow, and 15% more accurately on the reserve requirement than the current method. However, there is still a long way to implement this type of method in production, mainly due to risk and compliance concerns.

Conclusions

The utility industry is undergoing a paradigm shift. To be successful in this transition, it is necessary for utilities to fully utilize the data they own to transform the business in many ways. The following are some suggestions to be considered:

• Have a clear objective. Before initiating an analytics project, develop a business plan that defines the goals and the priorities clearly. Ask questions like, "What do we want to get out of the analytics?" "What data do we need?" "Who will be the users?" and "What is a reasonable timeline?" Also, involve users - both technical and non-technical - in the business plan development phase.
• Hire the right talent. Utility engineers who traditionally focus their jobs on technical studies may not possess the right data analytics skillset. Data scientists who have the right skills, on the other hand, usually lack knowledge of the industry and business content. The ideal candidates should have a power systems background and be savvy in analytical models, which normally require a solid understanding of statistics and some level programming. Like many other initiatives, a successful business transformation with data analytics is contingent on the company's culture. Creating an innovative environment that values experimentation and calculated risk-taking in the early stage of the analytics transformation can be particularly rewarding.
• Have a realistic expectation. Data analytics is not a silver bullet. It will provide insights to help reach an educated decision; however, it by no means is designed to answer every problem. Even the most powerful predictive algorithms today need some similar historical data patterns to learn from to produce a reasonable prediction. People who use analytics outputs blindly will eventually be shocked by the unanticipated (negative) consequences.
• One size doesn't fit all. Every industry is different; every company is different. Therefore, there is no one-size-fits-all solution. Instead, utilities should select the models and analyses that best fit their needs and capabilities. Some great off-the-shelf visualization tools such as Tableau and Microsoft BI can help utilities new to the field secure a quick win and a nice ROI. From there, they can build up their capabilities and advance their expertise to more sophisticated analytics.

Copyright by Hui Zhang, 2020, modified in 2022.