Title:On-Line Scheduling of a Residential Microgrid via Monte-Carlo Tree Search and a Learned Model

Abstract:The intermittent and uncertainty of distributed renewable energy brings significant challenges to reliable and economic operation of microgrids. Traditionally, on-line economic dispatch of microgrids is achieved by solving a linear or nonlinear optimization problem according to the ultra-short-term forecast information of the system. However, accurately forecasting the renewable power generations is still a tough task. To achieve on-line scheduling of a residential microgrid (RM) without any forecast information, this paper introduces a Model-Based Reinforcement Learning (MB-RL) based optimization strategy. The proposed online energy scheduling algorithm incorporates a learned model into Monte-Carlo tree search (MCTS), and can learn to improve the scheduling decisions off-line by self-play using the historical state data of the microgrid, then applied online to sequentially make operational decisions without relying on any forecast information. In each time period of the online optimization process, the optimal decision is obtained by conducting MCTS with a learned model that consists of the representation network, dynamic network, and prediction network. Especially, a new representation network architecture is designed in this paper considering the past renewable power generations and the load power that have a strong time correlation. In the representation network, long short-term memory (LSTM) units are adopted to extract features from the past load and renewable generation sequences, then the outputs of the LSTM units are fed into a fully connected neural network. The numerical simulation results demonstrate the effectiveness of the proposed algorithm.