-
Reinforcement Learning Optimizes Power Dispatch in Decentralized Power Grid
Authors:
Yongsun Lee,
Hoyun Choi,
Laurent Pagnier,
Cook Hyun Kim,
Jongshin Lee,
Bukyoung Jhun,
Heetae Kim,
Juergen Kurths,
B. Kahng
Abstract:
Effective frequency control in power grids has become increasingly important with the increasing demand for renewable energy sources. Here, we propose a novel strategy for resolving this challenge using graph convolutional proximal policy optimization (GC-PPO). The GC-PPO method can optimally determine how much power individual buses dispatch to reduce frequency fluctuations across a power grid. W…
▽ More
Effective frequency control in power grids has become increasingly important with the increasing demand for renewable energy sources. Here, we propose a novel strategy for resolving this challenge using graph convolutional proximal policy optimization (GC-PPO). The GC-PPO method can optimally determine how much power individual buses dispatch to reduce frequency fluctuations across a power grid. We demonstrate its efficacy in controlling disturbances by applying the GC-PPO to the power grid of the UK. The performance of GC-PPO is outstanding compared to the classical methods. This result highlights the promising role of GC-PPO in enhancing the stability and reliability of power systems by switching lines or decentralizing grid topology.
△ Less
Submitted 21 July, 2024;
originally announced July 2024.
-
Physics-Guided Actor-Critic Reinforcement Learning for Swimming in Turbulence
Authors:
Christopher Koh,
Laurent Pagnier,
Michael Chertkov
Abstract:
Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning strategy and comparing it with prescribed control and physics-agnostic reinforcement learning strategi…
▽ More
Turbulent diffusion causes particles placed in proximity to separate. We investigate the required swimming efforts to maintain an active particle close to its passively advected counterpart. We explore optimally balancing these efforts by developing a novel physics-informed reinforcement learning strategy and comparing it with prescribed control and physics-agnostic reinforcement learning strategies. Our scheme, coined the actor-physicist, is an adaptation of the actor-critic algorithm in which the neural network parameterized critic is replaced with an analytically derived physical heuristic function, the physicist. We validate the proposed physics-informed reinforcement learning approach through extensive numerical experiments in both synthetic BK and more realistic Arnold-Beltrami-Childress flow environments, demonstrating its superiority in controlling particle dynamics when compared to standard reinforcement learning methods.
△ Less
Submitted 10 November, 2024; v1 submitted 5 June, 2024;
originally announced June 2024.
-
System-Wide Emergency Policy for Transitioning from Main to Secondary Fuel
Authors:
Laurent Pagnier,
Criston Hyett,
Robert Ferrando,
Igal Goldshtein,
Jean Alisse,
Lilah Saban,
Michael Chertkov
Abstract:
Faced with the complexities of managing natural gas-dependent power system amid the surge of renewable integration and load unpredictability, this study explores strategies for navigating emergency transitions to costlier secondary fuels. Our aim is to develop decision-support tools for operators during such exigencies. We approach the problem through a Markov Decision Process (MDP) framework, acc…
▽ More
Faced with the complexities of managing natural gas-dependent power system amid the surge of renewable integration and load unpredictability, this study explores strategies for navigating emergency transitions to costlier secondary fuels. Our aim is to develop decision-support tools for operators during such exigencies. We approach the problem through a Markov Decision Process (MDP) framework, accounting for multiple uncertainties. These include the potential for dual-fuel generator failures and operator response during high-pressure situations. Additionally, we consider the finite reserves of primary fuel, governed by gas-flow partial differential equations (PDEs) and constrained by nodal pressure. Other factors include the variability in power forecasts due to renewable generation and the economic impact of compulsory load shedding. For tractability, we address the MDP in a simplified context, replacing it by Markov Processes evaluated against a selection of policies and scenarios for comparison. Our study considers two models for the natural gas system: an over-simplified model tracking linepack and a more nuanced model that accounts for gas flow network heterogeneity. The efficacy of our methods is demonstrated using a realistic model replicating Israel's power-gas infrastructure.
△ Less
Submitted 17 September, 2024; v1 submitted 14 November, 2023;
originally announced November 2023.
-
A Physics-Informed Machine Learning for Electricity Markets: A NYISO Case Study
Authors:
Robert Ferrando,
Laurent Pagnier,
Robert Mieth,
Zhirui Liang,
Yury Dvorkin,
Daniel Bienstock,
Michael Chertkov
Abstract:
This paper addresses the challenge of efficiently solving the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set lear…
▽ More
This paper addresses the challenge of efficiently solving the optimal power flow problem in real-time electricity markets. The proposed solution, named Physics-Informed Market-Aware Active Set learning OPF (PIMA-AS-OPF), leverages physical constraints and market properties to ensure physical and economic feasibility of market-clearing outcomes. Specifically, PIMA-AS-OPF employs the active set learning technique and expands its capabilities to account for curtailment in load or renewable power generation, which is a common challenge in real-world power systems. The core of PIMA-AS-OPF is a fully-connected neural network that takes the net load and the system topology as input. The outputs of this neural network include active constraints such as saturated generators and transmission lines, as well as non-zero load shedding and wind curtailments. These outputs allow for reducing the original market-clearing optimization to a system of linear equations, which can be solved efficiently and yield both the dispatch decisions and the locational marginal prices (LMPs). The dispatch decisions and LMPs are then tested for their feasibility with respect to the requirements for efficient market-clearing results. The accuracy and scalability of the proposed method is tested on a realistic 1814-bus NYISO system with current and future renewable energy penetration levels.
△ Less
Submitted 31 March, 2023;
originally announced April 2023.
-
Machine Learning for Electricity Market Clearing
Authors:
Laurent Pagnier,
Robert Ferrando,
Yury Dvorkin,
Michael Chertkov
Abstract:
This paper seeks to design a machine learning twin of the optimal power flow (OPF) optimization, which is used in market-clearing procedures by wholesale electricity markets. The motivation for the proposed approach stems from the need to obtain the digital twin, which is much faster than the original, while also being sufficiently accurate and producing consistent generation dispatches and locati…
▽ More
This paper seeks to design a machine learning twin of the optimal power flow (OPF) optimization, which is used in market-clearing procedures by wholesale electricity markets. The motivation for the proposed approach stems from the need to obtain the digital twin, which is much faster than the original, while also being sufficiently accurate and producing consistent generation dispatches and locational marginal prices (LMPs), which are primal and dual solutions of the OPF optimization, respectively. Availability of market-clearing tools based on this approach will enable computationally tractable evaluation of multiple dispatch scenarios under a given unit commitment. Rather than direct solution of OPF, the Karush-Kuhn-Tucker (KKT) conditions for the OPF problem in question may be written, and in parallel the LMPs of generators and loads may be expressed in terms of the OPF Lagrangian multipliers. Also, taking advantage of the practical fact that many of the Lagrangian multipliers associated with lines will be zero (thermal limits are not binding), we build and train an ML scheme which maps flexible resources (loads and renewables) to the binding lines, and supplement it with an efficient power-grid aware linear map to optimal dispatch and LMPs. The scheme is validated and illustrated on IEEE models. We also report a trade of analysis between quality of the reconstruction and number of samples needed to train the model.
△ Less
Submitted 23 May, 2022;
originally announced May 2022.
-
Towards Model Reduction for Power System Transients with Physics-Informed PDE
Authors:
Laurent Pagnier,
Michael Chertkov,
Julian Fritzsch,
Philippe Jacquod
Abstract:
This manuscript reports the first step towards building a robust and efficient model reduction methodology to capture transient dynamics in a transmission level electric power system. Such dynamics is normally modeled on seconds-to-tens-of-seconds time scales by the so-called swing equations, which are ordinary differential equations defined on a spatially discrete model of the power grid. Followi…
▽ More
This manuscript reports the first step towards building a robust and efficient model reduction methodology to capture transient dynamics in a transmission level electric power system. Such dynamics is normally modeled on seconds-to-tens-of-seconds time scales by the so-called swing equations, which are ordinary differential equations defined on a spatially discrete model of the power grid. Following Seymlyen (1974) and Thorpe, Seyler, and Phadke (1999), we suggest to map the swing equations onto a linear, inhomogeneous Partial Differential Equation (PDE) of parabolic type in two space and one time dimensions with time-independent coefficients and properly defined boundary conditions. We illustrate our method on the synchronous transmission grid of continental Europe. We show that, when properly coarse-grained, i.e., with the PDE coefficients and source terms extracted from a spatial convolution procedure of the respective discrete coefficients in the swing equations, the resulting PDE reproduces faithfully and efficiently the original swing dynamics. We finally discuss future extensions of this work, where the presented PDE-based modeling will initialize a physics-informed machine learning approach for real-time modeling, $n-1$ feasibility assessment and transient stability analysis of power systems.
△ Less
Submitted 11 July, 2022; v1 submitted 26 October, 2021;
originally announced October 2021.
-
Embedding Power Flow into Machine Learning for Parameter and State Estimation
Authors:
Laurent Pagnier,
Michael Chertkov
Abstract:
Modern state and parameter estimations in power systems consist of two stages: the outer problem of minimizing the mismatch between network observation and prediction over the network parameters, and the inner problem of predicting the system state for given values of the parameters. The standard solution of the combined problem is iterative: (a) set the parameters, e.g. to priors on the power lin…
▽ More
Modern state and parameter estimations in power systems consist of two stages: the outer problem of minimizing the mismatch between network observation and prediction over the network parameters, and the inner problem of predicting the system state for given values of the parameters. The standard solution of the combined problem is iterative: (a) set the parameters, e.g. to priors on the power line characteristics, (b) map input observation to prediction of the output, (c) compute the mismatch between predicted and observed output, (d) make a gradient descent step in the space of parameters to minimize the mismatch, and loop back to (a). We show how modern Machine Learning (ML), and specifically training guided by automatic differentiation, allows to resolve the iterative loop more efficiently. Moreover, we extend the scheme to the case of incomplete observations, where Phasor Measurement Units (reporting real and reactive powers, voltage and phase) are available only at the generators (PV buses), while loads (PQ buses) report (via SCADA controls) only active and reactive powers. Considering it from the implementation perspective, our methodology of resolving the parameter and state estimation problem can be viewed as embedding of the Power Flow (PF) solver into the training loop of the Machine Learning framework (PyTorch, in this study). We argue that this embedding can help to resolve high-level optimization problems in power system operations and planning.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
Physics-Informed Graphical Neural Network for Parameter & State Estimations in Power Systems
Authors:
Laurent Pagnier,
Michael Chertkov
Abstract:
Parameter Estimation (PE) and State Estimation (SE) are the most wide-spread tasks in the system engineering. They need to be done automatically, fast and frequently, as measurements arrive. Deep Learning (DL) holds the promise of tackling the challenge, however in so far, as PE and SE in power systems is concerned, (a) DL did not win trust of the system operators because of the lack of the physic…
▽ More
Parameter Estimation (PE) and State Estimation (SE) are the most wide-spread tasks in the system engineering. They need to be done automatically, fast and frequently, as measurements arrive. Deep Learning (DL) holds the promise of tackling the challenge, however in so far, as PE and SE in power systems is concerned, (a) DL did not win trust of the system operators because of the lack of the physics of electricity based, interpretations and (b) DL remained illusive in the operational regimes were data is scarce. To address this, we present a hybrid scheme which embeds physics modeling of power systems into Graphical Neural Networks (GNN), therefore empowering system operators with a reliable and explainable real-time predictions which can then be used to control the critical infrastructure. To enable progress towards trustworthy DL for PE and SE, we build a physics-informed method, named Power-GNN, which reconstructs physical, thus interpretable, parameters within Effective Power Flow (EPF) models, such as admittances of effective power lines, and NN parameters, representing implicitly unobserved elements of the system. In our experiments, we test the Power-GNN on different realistic power networks, including these with thousands of loads and hundreds of generators. We show that the Power-GNN outperforms vanilla NN scheme unaware of the EPF physics.
△ Less
Submitted 11 February, 2021;
originally announced February 2021.
-
Locating line and node disturbances in networks of diffusively coupled dynamical agents
Authors:
Robin Delabays,
Laurent Pagnier,
Melvyn Tyloo
Abstract:
A wide variety of natural and human-made systems consist of a large set of dynamical units coupled into a complex structure. Breakdown of such systems can have a dramatic impact, as in the case of neurons in the brain or lines in an electric grid, to name but a few. Preventing such catastrophic events requires in particular to be able to detect and locate the source of disturbances as fast as poss…
▽ More
A wide variety of natural and human-made systems consist of a large set of dynamical units coupled into a complex structure. Breakdown of such systems can have a dramatic impact, as in the case of neurons in the brain or lines in an electric grid, to name but a few. Preventing such catastrophic events requires in particular to be able to detect and locate the source of disturbances as fast as possible. We propose a simple method to identify and locate disturbances in networks of coupled dynamical agents, relying only on time series measurements and on the knowledge of the (possibly Kron-reduced) network structure. The strength and the appeal of the present approach lies in its simplicity paired with the ability to precisely locate disturbances and even to differentiate between line and node disturbances. If we have access to measurement at only a subset of nodes, our method is still able to identify the location of the disturbance if the disturbed nodes are measured. If not, we manage to identify the region of the network where the disturbance occurs.
△ Less
Submitted 7 April, 2021; v1 submitted 17 March, 2020;
originally announced March 2020.
-
Disturbance propagation, inertia location and slow modes in large-scale high voltage power grids
Authors:
Laurent Pagnier,
Philippe Jacquod
Abstract:
Conventional generators in power grids are steadily substituted with new renewable sources of electric power. The latter are connected to the grid via inverters and as such have little, if any rotational inertia. The resulting reduction of total inertia raises important issues of power grid stability, especially over short-time scales. We have constructed a model of the synchronous grid of contine…
▽ More
Conventional generators in power grids are steadily substituted with new renewable sources of electric power. The latter are connected to the grid via inverters and as such have little, if any rotational inertia. The resulting reduction of total inertia raises important issues of power grid stability, especially over short-time scales. We have constructed a model of the synchronous grid of continental Europe with which we numerically investigate frequency deviations as well as rates of change of frequency (RoCoF) following abrupt power losses. The magnitude of RoCoF's and frequency deviations strongly depend on the fault location, and we find the largest effects for faults located on the slowest mode - the Fiedler mode - of the network Laplacian matrix. This mode essentially vanishes over Belgium, Eastern France, Western Germany, northern Italy and Switzerland. Buses inside these regions are only weakly affected by faults occuring outside. Conversely, faults inside these regions have only a local effect and disturb only weakly outside buses. Following this observation, we reduce rotational inertia through three different procedures by either (i) reducing inertia on the Fiedler mode, (ii) reducing inertia homogeneously and (iii) reducing inertia outside the Fiedler mode. We find that procedure (iii) has little effect on disturbance propagation, while procedure (i) leads to the strongest increase of RoCoF and frequency deviations. These results for our model of the European transmission grid are corroborated by numerical investigations on the ERCOT transmission grid.
△ Less
Submitted 11 October, 2018;
originally announced October 2018.
