Showing 1–2 of 2 results for author: Badoual, M

A Learning-based Optimal Market Bidding Strategy for Price-Maker Energy Storage

Authors: Mathilde D. Badoual, Scott J. Moura

Abstract: Load serving entities with storage units reach sizes and performances that can significantly impact clearing prices in electricity markets. Nevertheless, price endogeneity is rarely considered in storage bidding strategies and modeling the electricity market is a challenging task. Meanwhile, model-free reinforcement learning such as the Actor-Critic are becoming increasingly popular for designing… ▽ More Load serving entities with storage units reach sizes and performances that can significantly impact clearing prices in electricity markets. Nevertheless, price endogeneity is rarely considered in storage bidding strategies and modeling the electricity market is a challenging task. Meanwhile, model-free reinforcement learning such as the Actor-Critic are becoming increasingly popular for designing energy system controllers. Yet implementation frequently requires lengthy, data-intense, and unsafe trial-and-error training. To fill these gaps, we implement an online Supervised Actor-Critic (SAC) algorithm, supervised with a model-based controller -- Model Predictive Control (MPC). The energy storage agent is trained with this algorithm to optimally bid while learning and adjusting to its impact on the market clearing prices. We compare the supervised Actor-Critic algorithm with the MPC algorithm as a supervisor, finding that the former reaps higher profits via learning. Our contribution, thus, is an online and safe SAC algorithm that outperforms the current model-based state-of-the-art. △ Less

Submitted 4 June, 2021; originally announced June 2021.

Comments: Presented at the 2021 American Control Conference (ACC), New Orleans, USA, May 25-28, 2021

Bayesian Hierarchical Methods for Modeling Electrical Grid Component Failures

Authors: Laurel N. Dunn, Ioanna Kavvada, Mathilde Badoual, Scott J. Moura

Abstract: Failure probabilities for grid components are often estimated using parametric models which can capitalize on operational grid data. This work formulates a Bayesian hierarchical framework designed to integrate data and domain expertise to understand the failure properties of a regional power system, where variability in the expected performance of individual components gives rise to failure proces… ▽ More Failure probabilities for grid components are often estimated using parametric models which can capitalize on operational grid data. This work formulates a Bayesian hierarchical framework designed to integrate data and domain expertise to understand the failure properties of a regional power system, where variability in the expected performance of individual components gives rise to failure processes that are heterogeneous and uncertain. We use Bayesian methods to fit failure models to failure data generated in simulation. We test our algorithm by evaluating differences between the data-generating model, our Bayesian hierarchical model, and maximum likelihood parameter estimates. We evaluate how well each model can approximate the failure properties of individual components, and of the system overall. Finally, we define an upgrade policy for achieving targeted reductions in risk exposure, and compare the magnitude of upgrades recommended by each model. △ Less

Submitted 21 January, 2020; originally announced January 2020.