-
CMIP X-MOS: Improving Climate Models with Extreme Model Output Statistics
Authors:
Vsevolod Morozov,
Artem Galliamov,
Aleksandr Lukashevich,
Antonina Kurdukova,
Yury Maximov
Abstract:
Climate models are essential for assessing the impact of greenhouse gas emissions on our changing climate and the resulting increase in the frequency and severity of natural disasters. Despite the widespread acceptance of climate models produced by the Coupled Model Intercomparison Project (CMIP), they still face challenges in accurately predicting climate extremes, which pose most significant thr…
▽ More
Climate models are essential for assessing the impact of greenhouse gas emissions on our changing climate and the resulting increase in the frequency and severity of natural disasters. Despite the widespread acceptance of climate models produced by the Coupled Model Intercomparison Project (CMIP), they still face challenges in accurately predicting climate extremes, which pose most significant threats to both people and the environment. To address this limitation and improve predictions of natural disaster risks, we introduce Extreme Model Output Statistics (X-MOS). This approach utilizes deep regression techniques to precisely map CMIP model outputs to real measurements obtained from weather stations, which results in a more accurate analysis of the XXI climate extremes. In contrast to previous research, our study places a strong emphasis on enhancing the estimation of the tails of future climate parameter distributions. The latter supports decision-makers, enabling them to better assess climate-related risks across the globe.
△ Less
Submitted 24 October, 2023;
originally announced November 2023.
-
Climate Change Impact on Agricultural Land Suitability: An Interpretable Machine Learning-Based Eurasia Case Study
Authors:
Valeriy Shevchenko,
Daria Taniushkina,
Aleksander Lukashevich,
Aleksandr Bulkin,
Roland Grinis,
Kirill Kovalev,
Veronika Narozhnaia,
Nazar Sotiriadi,
Alexander Krenke,
Yury Maximov
Abstract:
The United Nations has identified improving food security and reducing hunger as essential components of its sustainable development goals. As of 2021, approximately 828 million people worldwide are experiencing hunger and malnutrition, with numerous fatalities reported. Climate change significantly impacts agricultural land suitability, potentially leading to severe food shortages and subsequent…
▽ More
The United Nations has identified improving food security and reducing hunger as essential components of its sustainable development goals. As of 2021, approximately 828 million people worldwide are experiencing hunger and malnutrition, with numerous fatalities reported. Climate change significantly impacts agricultural land suitability, potentially leading to severe food shortages and subsequent social and political conflicts. To address this pressing issue, we have developed a machine learning-based approach to predict the risk of substantial land suitability degradation and changes in irrigation patterns. Our study focuses on Central Eurasia, a region burdened with economic and social challenges.
This study represents a pioneering effort in utilizing machine learning methods to assess the impact of climate change on agricultural land suitability under various carbon emissions scenarios. Through comprehensive feature importance analysis, we unveil specific climate and terrain characteristics that exert influence on land suitability. Our approach achieves remarkable accuracy, offering policymakers invaluable insights to facilitate informed decisions aimed at averting a humanitarian crisis, including strategies such as the provision of additional water and fertilizers. This research underscores the tremendous potential of machine learning in addressing global challenges, with a particular emphasis on mitigating hunger and malnutrition.
△ Less
Submitted 24 October, 2023;
originally announced October 2023.
-
A-Priori Reduction of Scenario Approximation for Automated Generation Control in High-Voltage Power Grids with Renewable Energy
Authors:
Aleksander Lukashevich,
Aleksander Bulkin,
Yury Maximov
Abstract:
Renewable energy sources (RES) are increasingly integrated into power systems to support the United Nations' Sustainable Development Goals of decarbonization and energy security. However, their low inertia and high uncertainty pose challenges to grid stability and increase the risk of blackouts. Stochastic chance-constrained optimization, particularly data-driven methods, offers solutions but can…
▽ More
Renewable energy sources (RES) are increasingly integrated into power systems to support the United Nations' Sustainable Development Goals of decarbonization and energy security. However, their low inertia and high uncertainty pose challenges to grid stability and increase the risk of blackouts. Stochastic chance-constrained optimization, particularly data-driven methods, offers solutions but can be time-consuming, especially when handling multiple system snapshots. This paper addresses a dynamic joint chance-constrained Direct Current Optimal Power Flow (DC-OPF) problem with Automated Generation Control (AGC) to facilitate cost-effective power generation while ensuring that balance and security constraints are met. We propose an approach for a data-driven approximation that includes a priori sample reduction, maintaining solution reliability while reducing the size of the data-driven approximation. Both theoretical analysis and empirical results demonstrate the superiority of this approach in handling generation uncertainty, requiring up to twice less data while preserving solution reliability.
△ Less
Submitted 5 October, 2024; v1 submitted 3 October, 2023;
originally announced October 2023.
-
GP CC-OPF: Gaussian Process based optimization tool for Chance-Constrained Optimal Power Flow
Authors:
Mile Mitrovic,
Ognjen Kundacina,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Yury Maximov,
Deepjyoti Deka
Abstract:
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based…
▽ More
The Gaussian Process (GP) based Chance-Constrained Optimal Power Flow (CC-OPF) is an open-source Python code developed for solving economic dispatch (ED) problem in modern power grids. In recent years, integrating a significant amount of renewables into a power grid causes high fluctuations and thus brings a lot of uncertainty to power grid operations. This fact makes the conventional model-based CC-OPF problem non-convex and computationally complex to solve. The developed tool presents a novel data-driven approach based on the GP regression model for solving the CC-OPF problem with a trade-off between complexity and accuracy. The proposed approach and developed software can help system operators to effectively perform ED optimization in the presence of large uncertainties in the power grid.
△ Less
Submitted 16 February, 2023;
originally announced February 2023.
-
Predicting spatial distribution of Palmer Drought Severity Index
Authors:
V. Grabar,
A. Lukashevich,
A. Zaytsev
Abstract:
The probability of a drought for a particular region is crucial when making decisions related to agriculture. Forecasting this probability is critical for management and challenging at the same time. The prediction model should consider multiple factors with complex relationships across the region of interest and neighbouring regions.
We approach this problem by presenting an end-to-end solution…
▽ More
The probability of a drought for a particular region is crucial when making decisions related to agriculture. Forecasting this probability is critical for management and challenging at the same time. The prediction model should consider multiple factors with complex relationships across the region of interest and neighbouring regions.
We approach this problem by presenting an end-to-end solution based on a spatio-temporal neural network. The model predicts the Palmer Drought Severity Index (PDSI) for subregions of interest. Predictions by climate models provide an additional source of knowledge of the model leading to more accurate drought predictions.
Our model has better accuracy than baseline Gradient boosting solutions, as the $R^2$ score for it is $0.90$ compared to $0.85$ for Gradient boosting. Specific attention is on the range of applicability of the model. We examine various regions across the globe to validate them under different conditions.
We complement the results with an analysis of how future climate changes for different scenarios affect the PDSI and how our model can help to make better decisions and more sustainable economics.
△ Less
Submitted 1 September, 2022; v1 submitted 31 August, 2022;
originally announced August 2022.
-
Data-Driven Chance Constrained AC-OPF using Hybrid Sparse Gaussian Processes
Authors:
Mile Mitrovic,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Yury Maximov,
Deepjyoti Deka
Abstract:
The alternating current (AC) chance-constrained optimal power flow (CC-OPF) problem addresses the economic efficiency of electricity generation and delivery under generation uncertainty. The latter is intrinsic to modern power grids because of the high amount of renewables. Despite its academic success, the AC CC-OPF problem is highly nonlinear and computationally demanding, which limits its pract…
▽ More
The alternating current (AC) chance-constrained optimal power flow (CC-OPF) problem addresses the economic efficiency of electricity generation and delivery under generation uncertainty. The latter is intrinsic to modern power grids because of the high amount of renewables. Despite its academic success, the AC CC-OPF problem is highly nonlinear and computationally demanding, which limits its practical impact. For improving the AC-OPF problem complexity/accuracy trade-off, the paper proposes a fast data-driven setup that uses the sparse and hybrid Gaussian processes (GP) framework to model the power flow equations with input uncertainty. We advocate the efficiency of the proposed approach by a numerical study over multiple IEEE test cases showing up to two times faster and more accurate solutions compared to the state-of-the-art methods.
△ Less
Submitted 30 August, 2022;
originally announced August 2022.
-
Data-Driven Stochastic AC-OPF using Gaussian Processes
Authors:
Mile Mitrovic,
Aleksandr Lukashevich,
Petr Vorobev,
Vladimir Terzija,
Semen Budenny,
Yury Maximov,
Deepjyoti Deka
Abstract:
In recent years, electricity generation has been responsible for more than a quarter of the greenhouse gas emissions in the US. Integrating a significant amount of renewables into a power grid is probably the most accessible way to reduce carbon emissions from power grids and slow down climate change. Unfortunately, the most accessible renewable power sources, such as wind and solar, are highly fl…
▽ More
In recent years, electricity generation has been responsible for more than a quarter of the greenhouse gas emissions in the US. Integrating a significant amount of renewables into a power grid is probably the most accessible way to reduce carbon emissions from power grids and slow down climate change. Unfortunately, the most accessible renewable power sources, such as wind and solar, are highly fluctuating and thus bring a lot of uncertainty to power grid operations and challenge existing optimization and control policies. The chance-constrained alternating current (AC) optimal power flow (OPF) framework finds the minimum cost generation dispatch maintaining the power grid operations within security limits with a prescribed probability. Unfortunately, the AC-OPF problem's chance-constrained extension is non-convex, computationally challenging, and requires knowledge of system parameters and additional assumptions on the behavior of renewable distribution. Known linear and convex approximations to the above problems, though tractable, are too conservative for operational practice and do not consider uncertainty in system parameters. This paper presents an alternative data-driven approach based on Gaussian process (GP) regression to close this gap. The GP approach learns a simple yet non-convex data-driven approximation to the AC power flow equations that can incorporate uncertainty inputs. The latter is then used to determine the solution of CC-OPF efficiently, by accounting for both input and parameter uncertainty. The practical efficiency of the proposed approach using different approximations for GP-uncertainty propagation is illustrated over numerous IEEE test cases.
△ Less
Submitted 21 July, 2022;
originally announced July 2022.
-
Importance sampling approach to chance-constrained DC optimal power flow
Authors:
Aleksander Lukashevich,
Vyacheslav Gorchakov,
Petr Vorobev,
Deepjyoti Deka,
Yury Maximov
Abstract:
Despite significant economic and ecological effects, a higher level of renewable energy generation leads to increased uncertainty and variability in power injections, thus compromising grid reliability. In order to improve power grid security, we investigate a joint chance-constrained (CC) direct current (DC) optimal power flow (OPF) problem. The problem aims to find economically optimal power gen…
▽ More
Despite significant economic and ecological effects, a higher level of renewable energy generation leads to increased uncertainty and variability in power injections, thus compromising grid reliability. In order to improve power grid security, we investigate a joint chance-constrained (CC) direct current (DC) optimal power flow (OPF) problem. The problem aims to find economically optimal power generation while guaranteeing that all power generation, line flows, and voltages simultaneously remain within their bounds with a pre-defined probability. Unfortunately, the problem is computationally intractable even if the distribution of renewables fluctuations is specified. Moreover, existing approximate solutions to the joint CC OPF problem are overly conservative, and therefore have less value for the operational practice. This paper proposes an importance sampling approach to the CC DC OPF problem, which yields better complexity and accuracy than current state-of-the-art methods. The algorithm efficiently reduces the number of scenarios by generating and using only the most important of them, thus enabling real-time solutions for test cases with up to several hundred buses.
△ Less
Submitted 23 November, 2021;
originally announced November 2021.
-
Polarizabilities of microparticles in relativistic field theory
Authors:
N. V. Maksimenko,
O. M. Deruzhkova,
S. A. Lukashevich
Abstract:
The article determine a relativistic tensor of the second rank containing the vectors of the electric and magnetic polarization of the medium, based on the Maxwell equations and the determination of the charge density and polarization current of a structural microparticle. Using this tensor and the electromagnetic field tensor, the relativistic lagrangian of the interaction an electromagnetic fiel…
▽ More
The article determine a relativistic tensor of the second rank containing the vectors of the electric and magnetic polarization of the medium, based on the Maxwell equations and the determination of the charge density and polarization current of a structural microparticle. Using this tensor and the electromagnetic field tensor, the relativistic lagrangian of the interaction an electromagnetic field with a structural microparticle is obtained, taking into account the polarization of its structural elements. Based on the induced moments, the relativistic lagrangian of the interaction an electromagnetic field with a structural microparticle is constructed with considering the electric and magnetic polarizabilities.
△ Less
Submitted 4 May, 2022; v1 submitted 21 August, 2021;
originally announced August 2021.
-
Power Grid Reliability Estimation via Adaptive Importance Sampling
Authors:
Aleksandr Lukashevich,
Yury Maximov
Abstract:
Electricity production currently generates approximately 25% of greenhouse gas emissions in the USA. Thus, increasing the amount of renewable energy is a key step to carbon neutrality. However, integrating a large amount of fluctuating renewable generation is a significant challenge for power grid operating and planning. Grid reliability, i.e., an ability to meet operational constraints under powe…
▽ More
Electricity production currently generates approximately 25% of greenhouse gas emissions in the USA. Thus, increasing the amount of renewable energy is a key step to carbon neutrality. However, integrating a large amount of fluctuating renewable generation is a significant challenge for power grid operating and planning. Grid reliability, i.e., an ability to meet operational constraints under power fluctuations, is probably the most important of them. In this paper, we propose computationally efficient and accurate methods to estimate the probability of failure, i.e. reliability constraints violation, under a known distribution of renewable energy generation. To this end, we investigate an importance sampling approach, a flexible extension of Monte-Carlo methods, which adaptively changes the sampling distribution to generate more samples near the reliability boundary. The approach allows to estimate failure probability in real-time based only on a few dozens of random samples, compared to thousands required by the plain Monte-Carlo. Our study focuses on high voltage direct current power transmission grids with linear reliability constraints on power injections and line currents. We propose a novel theoretically justified physics-informed adaptive importance sampling algorithm and compare its performance to state-of-the-art methods on multiple IEEE power grid test cases.
△ Less
Submitted 18 May, 2021;
originally announced May 2021.
-
An Accelerated Second-Order Method for Distributed Stochastic Optimization
Authors:
Artem Agafonov,
Pavel Dvurechensky,
Gesualdo Scutari,
Alexander Gasnikov,
Dmitry Kamzolov,
Aleksandr Lukashevich,
Amir Daneshmand
Abstract:
We consider distributed stochastic optimization problems that are solved with master/workers computation architecture. Statistical arguments allow to exploit statistical similarity and approximate this problem by a finite-sum problem, for which we propose an inexact accelerated cubic-regularized Newton's method that achieves lower communication complexity bound for this setting and improves upon e…
▽ More
We consider distributed stochastic optimization problems that are solved with master/workers computation architecture. Statistical arguments allow to exploit statistical similarity and approximate this problem by a finite-sum problem, for which we propose an inexact accelerated cubic-regularized Newton's method that achieves lower communication complexity bound for this setting and improves upon existing upper bound. We further exploit this algorithm to obtain convergence rate bounds for the original stochastic optimization problem and compare our bounds with the existing bounds in several regimes when the goal is to minimize the number of communication rounds and increase the parallelization by increasing the number of workers.
△ Less
Submitted 26 March, 2021;
originally announced March 2021.
-
Hyperfast Second-Order Local Solvers for Efficient Statistically Preconditioned Distributed Optimization
Authors:
Pavel Dvurechensky,
Dmitry Kamzolov,
Aleksandr Lukashevich,
Soomin Lee,
Erik Ordentlich,
César A. Uribe,
Alexander Gasnikov
Abstract:
Statistical preconditioning enables fast methods for distributed large-scale empirical risk minimization problems. In this approach, multiple worker nodes compute gradients in parallel, which are then used by the central node to update the parameter by solving an auxiliary (preconditioned) smaller-scale optimization problem. The recently proposed Statistically Preconditioned Accelerated Gradient (…
▽ More
Statistical preconditioning enables fast methods for distributed large-scale empirical risk minimization problems. In this approach, multiple worker nodes compute gradients in parallel, which are then used by the central node to update the parameter by solving an auxiliary (preconditioned) smaller-scale optimization problem. The recently proposed Statistically Preconditioned Accelerated Gradient (SPAG) method has complexity bounds superior to other such algorithms but requires an exact solution for computationally intensive auxiliary optimization problems at every iteration. In this paper, we propose an Inexact SPAG (InSPAG) and explicitly characterize the accuracy by which the corresponding auxiliary subproblem needs to be solved to guarantee the same convergence rate as the exact method. We build our results by first developing an inexact adaptive accelerated Bregman proximal gradient method for general optimization problems under relative smoothness and strong convexity assumptions, which may be of independent interest. Moreover, we explore the properties of the auxiliary problem in the InSPAG algorithm assuming Lipschitz third-order derivatives and strong convexity. For such problem class, we develop a linearly convergent Hyperfast second-order method and estimate the total complexity of the InSPAG method with hyperfast auxiliary problem solver. Finally, we illustrate the proposed method's practical efficiency by performing large-scale numerical experiments on logistic regression models. To the best of our knowledge, these are the first empirical results on implementing high-order methods on large-scale problems, we work with data where the dimension is of the order of~$3$ million, and the number of samples is~$700$ million.
△ Less
Submitted 3 October, 2022; v1 submitted 16 February, 2021;
originally announced February 2021.
