-
Reinforcement Learning and Mixed-Integer Programming for Power Plant Scheduling in Low Carbon Systems: Comparison and Hybridisation
Authors:
Cormac O'Malley,
Patrick de Mars,
Luis Badesa,
Goran Strbac
Abstract:
Decarbonisation is driving dramatic growth in renewable power generation. This increases uncertainty in the load to be served by power plants and makes their efficient scheduling, known as the unit commitment (UC) problem, more difficult. UC is solved in practice by mixed-integer programming (MIP) methods; however, there is growing interest in emerging data-driven methods including reinforcement l…
▽ More
Decarbonisation is driving dramatic growth in renewable power generation. This increases uncertainty in the load to be served by power plants and makes their efficient scheduling, known as the unit commitment (UC) problem, more difficult. UC is solved in practice by mixed-integer programming (MIP) methods; however, there is growing interest in emerging data-driven methods including reinforcement learning (RL). In this paper, we extensively test two MIP (deterministic and stochastic) and two RL (model-free and with lookahead) scheduling methods over a large set of test days and problem sizes, for the first time comparing the state-of-the-art of these two approaches on a level playing field. We find that deterministic and stochastic MIP consistently produce lower-cost UC schedules than RL, exhibiting better reliability and scalability with problem size. Average operating costs of RL are more than 2 times larger than stochastic MIP for a 50-generator test case, while the cost is 13 times larger in the worst instance. However, the key strength of RL is the ability to produce solutions practically instantly, irrespective of problem size. We leverage this advantage to produce various initial solutions for warm starting concurrent stochastic MIP solves. By producing several near-optimal solutions simultaneously and then evaluating them using Monte Carlo methods, the differences between the true cost function and the discrete approximation required to formulate the MIP are exploited. The resulting hybrid technique outperforms both the RL and MIP methods individually, reducing total operating costs by 0.3% on average.
△ Less
Submitted 9 December, 2022;
originally announced December 2022.
-
Chance-constrained allocation of UFLS candidate feeders under high penetration of distributed generation
Authors:
Luis Badesa,
Cormac O'Malley,
Maria Parajeles,
Goran Strbac
Abstract:
Under-Frequency Load Shedding (UFLS) schemes are the last resort to contain a frequency drop in the grid by disconnecting part of the demand. The allocation methods for selecting feeders that would contribute to the UFLS scheme have traditionally relied on the fact that electric demand followed fairly regular patterns, and could be forecast with high accuracy. However, recent integration of Distri…
▽ More
Under-Frequency Load Shedding (UFLS) schemes are the last resort to contain a frequency drop in the grid by disconnecting part of the demand. The allocation methods for selecting feeders that would contribute to the UFLS scheme have traditionally relied on the fact that electric demand followed fairly regular patterns, and could be forecast with high accuracy. However, recent integration of Distributed Generation (DG) increases the uncertainty in net consumption of feeders which, in turn, requires a reformulation of UFLS-allocation methods to account for this uncertainty. In this paper, a chance-constrained methodology for selecting feeders is proposed, with mathematical guarantees for the disconnection of the required amount of load with a certain pre-defined probability. The correlation in net-load forecasts among feeders is explicitly considered, given that uncertainty in DG power output is driven by meteorological conditions with high correlation across the network. Furthermore, this method is applicable either to systems with conventional UFLS schemes (where relays measure local frequency and trip if this magnitude falls below a certain threshold), or adaptive UFLS schemes (where relays are triggered by control signals sent in the few instants following a contingency). Relevant case studies demonstrate the applicability of the proposed method, and the need for explicit consideration of uncertainty in the UFLS-allocation process.
△ Less
Submitted 5 November, 2022;
originally announced November 2022.
-
Frequency Response from Aggregated V2G Chargers With Uncertain EV Connections
Authors:
Cormac O'Malley,
Luis Badesa,
Fei Teng,
Goran Strbac
Abstract:
Fast frequency response (FR) is highly effective at securing frequency dynamics after a generator outage in low inertia systems. Electric vehicles (EVs) equipped with vehicle to grid (V2G) chargers could offer an abundant source of FR in future. However, the uncertainty associated with V2G aggregation, driven by the uncertain number of connected EVs at the time of an outage, has not been fully und…
▽ More
Fast frequency response (FR) is highly effective at securing frequency dynamics after a generator outage in low inertia systems. Electric vehicles (EVs) equipped with vehicle to grid (V2G) chargers could offer an abundant source of FR in future. However, the uncertainty associated with V2G aggregation, driven by the uncertain number of connected EVs at the time of an outage, has not been fully understood and prevents its participation in the existing service provision framework. To tackle this limitation, this paper, for the first time, incorporates such uncertainty into system frequency dynamics, from which probabilistic nadir and steady state frequency requirements are enforced via a derived momeent-based distributionally-robust chance constraint. Field data from over 25,000 chargers is analysed to provide realistic parameters and connection forecasts to examine the value of FR from V2G chargers in annual operation of the the GB 2030 system. The case study demonstrates that uncertainty of EV connections can be effectively managed through the proposed scheduling framework, which results in annual savings of £6,300 or 37.4 tCO2 per charger. The sensitivity of this value to renewable capacity and FR delays is explored, with V2G capacity shown to be a third as valuable as the same grid battery capacity.
△ Less
Submitted 26 August, 2022; v1 submitted 16 February, 2022;
originally announced February 2022.
-
Value of Fleet Vehicle Grid in Providing Transmission System Operator Services
Authors:
Cormac O'Malley,
Marko Aunedi,
Fei Teng,
Goran Strbac
Abstract:
In this paper a new aggregated model for electric vehicle (EV) fleets is presented that considers their daily and weekly usage patterns. A frequency-constrained stochastic unit commitment model is employed to optimally schedule EV charging and discharging as well as the provision of frequency response (FR) in an electricity system, while respecting the vehicles' energy requirements and driving sch…
▽ More
In this paper a new aggregated model for electric vehicle (EV) fleets is presented that considers their daily and weekly usage patterns. A frequency-constrained stochastic unit commitment model is employed to optimally schedule EV charging and discharging as well as the provision of frequency response (FR) in an electricity system, while respecting the vehicles' energy requirements and driving schedules. Through case studies we demonstrate that an EV with vehicle to grid (V2G) capability can reduce system costs in a future GB electricity grid by up to £12,000 per year, and reduce CO2 emissions by 60 tonnes per year, mainly due to reduced curtailment of wind power. The paper also quantifies the changes in the benefits of fleet V2G resulting from variations in FR delivery time, the penetration of wind or uptake of alternative flexibility providers. Finally , a battery degradation model dependent on an EV's state of charge is proposed and implemented in the stochastic scheduling problem. It enables significant degradation cost reductions of 16% with only a 0.4% reduction of an EV's system value.
△ Less
Submitted 14 January, 2022;
originally announced January 2022.
-
Probabilistic Scheduling of UFLS to Secure Credible Contingencies in Low Inertia Systems
Authors:
Cormac O'Malley,
Luis Badesa,
Fei Teng,
Goran Strbac
Abstract:
The reduced inertia levels in low-carbon power grids necessitate explicit constraints to limit frequency's nadir and rate of change during scheduling. This can result in significant curtailment of renewable energy due to the minimum generation of thermal plants that are needed to provide frequency response (FR) and inertia. Additional consideration of fast FR, a dynamically reduced largest loss an…
▽ More
The reduced inertia levels in low-carbon power grids necessitate explicit constraints to limit frequency's nadir and rate of change during scheduling. This can result in significant curtailment of renewable energy due to the minimum generation of thermal plants that are needed to provide frequency response (FR) and inertia. Additional consideration of fast FR, a dynamically reduced largest loss and under frequency load shedding (UFLS) allows frequency security to be achieved more cost effectively. This paper derives a novel nadir constraint from the swing equation that, for the first time, provides a framework for the optimal comparison of all these services. We demonstrate that this constraint can be accurately and conservatively approximated for moderate UFLS levels with a second order cone, resulting in highly tractable convex problems. Case studies performed on a Great Britain 2030 system demonstrate that UFLS as an option to contain single plant outages can reduce annual operational costs by up to £559m, 52% of frequency security costs. The sensitivity of this value to wind penetration, abundance of alternative frequency services, UFLS amount and cost is explored.
△ Less
Submitted 17 November, 2021; v1 submitted 11 March, 2021;
originally announced March 2021.
