-
Extrapolating the long-term seasonal component of electricity prices for forecasting in the day-ahead market
Authors:
Katarzyna Chęć,
Bartosz Uniejewski,
Rafał Weron
Abstract:
Recent studies provide evidence that decomposing the electricity price into the long-term seasonal component (LTSC) and the remaining part, predicting both separately, and then combining their forecasts can bring significant accuracy gains in day-ahead electricity price forecasting. However, not much attention has been paid to predicting the LTSC, and the last 24 hourly values of the estimated pat…
▽ More
Recent studies provide evidence that decomposing the electricity price into the long-term seasonal component (LTSC) and the remaining part, predicting both separately, and then combining their forecasts can bring significant accuracy gains in day-ahead electricity price forecasting. However, not much attention has been paid to predicting the LTSC, and the last 24 hourly values of the estimated pattern are typically copied for the target day. To address this gap, we introduce a novel approach which extracts the trend-seasonal pattern from a price series extrapolated using price forecasts for the next 24 hours. We assess it using two 5-year long test periods from the German and Spanish power markets, covering the Covid-19 pandemic, the 2021/2022 energy crisis, and the war in Ukraine. Considering parsimonious autoregressive and LASSO-estimated models, we find that improvements in predictive accuracy range from 3\% to 15\% in terms of the root mean squared error and exceed 1\% in terms of profits from a realistic trading strategy involving day-ahead bidding and battery storage.
△ Less
Submitted 4 March, 2025;
originally announced March 2025.
-
Postprocessing of point predictions for probabilistic forecasting of day-ahead electricity prices: The benefits of using isotonic distributional regression
Authors:
Arkadiusz Lipiecki,
Bartosz Uniejewski,
Rafał Weron
Abstract:
Operational decisions relying on predictive distributions of electricity prices can result in significantly higher profits compared to those based solely on point forecasts. However, the majority of models developed in both academic and industrial settings provide only point predictions. To address this, we examine three postprocessing methods for converting point forecasts of day-ahead electricit…
▽ More
Operational decisions relying on predictive distributions of electricity prices can result in significantly higher profits compared to those based solely on point forecasts. However, the majority of models developed in both academic and industrial settings provide only point predictions. To address this, we examine three postprocessing methods for converting point forecasts of day-ahead electricity prices into probabilistic ones: Quantile Regression Averaging, Conformal Prediction, and the recently introduced Isotonic Distributional Regression. We find that while the latter demonstrates the most varied behavior, it contributes the most to the ensemble of the three predictive distributions, as measured by Shapley values. Remarkably, the performance of the combination is superior to that of state-of-the-art Distributional Deep Neural Networks over two 4.5-year test periods from the German and Spanish power markets, spanning the COVID pandemic and the war in Ukraine.
△ Less
Submitted 6 October, 2024; v1 submitted 2 April, 2024;
originally announced April 2024.
-
Combining predictive distributions of electricity prices: Does minimizing the CRPS lead to optimal decisions in day-ahead bidding?
Authors:
Weronika Nitka,
Rafał Weron
Abstract:
Probabilistic price forecasting has recently gained attention in power trading because decisions based on such predictions can yield significantly higher profits than those made with point forecasts alone. At the same time, methods are being developed to combine predictive distributions, since no model is perfect and averaging generally improves forecasting performance. In this article we address…
▽ More
Probabilistic price forecasting has recently gained attention in power trading because decisions based on such predictions can yield significantly higher profits than those made with point forecasts alone. At the same time, methods are being developed to combine predictive distributions, since no model is perfect and averaging generally improves forecasting performance. In this article we address the question of whether using CRPS learning, a novel weighting technique minimizing the continuous ranked probability score (CRPS), leads to optimal decisions in day-ahead bidding. To this end, we conduct an empirical study using hourly day-ahead electricity prices from the German EPEX market. We find that increasing the diversity of an ensemble can have a positive impact on accuracy. At the same time, the higher computational cost of using CRPS learning compared to an equal-weighted aggregation of distributions is not offset by higher profits, despite significantly more accurate predictions.
△ Less
Submitted 29 August, 2023;
originally announced August 2023.
-
Distributional neural networks for electricity price forecasting
Authors:
Grzegorz Marcjasz,
Michał Narajewski,
Rafał Weron,
Florian Ziel
Abstract:
We present a novel approach to probabilistic electricity price forecasting which utilizes distributional neural networks. The model structure is based on a deep neural network that contains a so-called probability layer. The network's output is a parametric distribution with 2 (normal) or 4 (Johnson's SU) parameters. In a forecasting study involving day-ahead electricity prices in the German marke…
▽ More
We present a novel approach to probabilistic electricity price forecasting which utilizes distributional neural networks. The model structure is based on a deep neural network that contains a so-called probability layer. The network's output is a parametric distribution with 2 (normal) or 4 (Johnson's SU) parameters. In a forecasting study involving day-ahead electricity prices in the German market, our approach significantly outperforms state-of-the-art benchmarks, including LASSO-estimated regressions and deep neural networks combined with Quantile Regression Averaging. The obtained results not only emphasize the importance of higher moments when modeling volatile electricity prices, but also -- given that probabilistic forecasting is the essence of risk management -- provide important implications for managing portfolios in the power sector.
△ Less
Submitted 10 December, 2022; v1 submitted 6 July, 2022;
originally announced July 2022.
-
Forecasting Electricity Prices
Authors:
Katarzyna Maciejowska,
Bartosz Uniejewski,
Rafał Weron
Abstract:
Forecasting electricity prices is a challenging task and an active area of research since the 1990s and the deregulation of the traditionally monopolistic and government-controlled power sectors. Although it aims at predicting both spot and forward prices, the vast majority of research is focused on short-term horizons which exhibit dynamics unlike in any other market. The reason is that power sys…
▽ More
Forecasting electricity prices is a challenging task and an active area of research since the 1990s and the deregulation of the traditionally monopolistic and government-controlled power sectors. Although it aims at predicting both spot and forward prices, the vast majority of research is focused on short-term horizons which exhibit dynamics unlike in any other market. The reason is that power system stability calls for a constant balance between production and consumption, while being weather (both demand and supply) and business activity (demand only) dependent. The recent market innovations do not help in this respect. The rapid expansion of intermittent renewable energy sources is not offset by the costly increase of electricity storage capacities and modernization of the grid infrastructure. On the methodological side, this leads to three visible trends in electricity price forecasting research as of 2022. Firstly, there is a slow, but more noticeable with every year, tendency to consider not only point but also probabilistic (interval, density) or even path (also called ensemble) forecasts. Secondly, there is a clear shift from the relatively parsimonious econometric (or statistical) models towards more complex and harder to comprehend, but more versatile and eventually more accurate statistical/machine learning approaches. Thirdly, statistical error measures are nowadays regarded as only the first evaluation step. Since they may not necessarily reflect the economic value of reducing prediction errors, more and more often, they are complemented by case studies comparing profits from scheduling or trading strategies based on price forecasts obtained from different models.
△ Less
Submitted 25 April, 2022;
originally announced April 2022.
-
Electricity Price Forecasting: The Dawn of Machine Learning
Authors:
Arkadiusz Jędrzejewski,
Jesus Lago,
Grzegorz Marcjasz,
Rafał Weron
Abstract:
Electricity price forecasting (EPF) is a branch of forecasting on the interface of electrical engineering, statistics, computer science, and finance, which focuses on predicting prices in wholesale electricity markets for a whole spectrum of horizons. These range from a few minutes (real-time/intraday auctions and continuous trading), through days (day-ahead auctions), to weeks, months or even yea…
▽ More
Electricity price forecasting (EPF) is a branch of forecasting on the interface of electrical engineering, statistics, computer science, and finance, which focuses on predicting prices in wholesale electricity markets for a whole spectrum of horizons. These range from a few minutes (real-time/intraday auctions and continuous trading), through days (day-ahead auctions), to weeks, months or even years (exchange and over-the-counter traded futures and forward contracts). Over the last 25 years, various methods and computational tools have been applied to intraday and day-ahead EPF. Until the early 2010s, the field was dominated by relatively small linear regression models and (artificial) neural networks, typically with no more than two dozen inputs. As time passed, more data and more computational power became available. The models grew larger to the extent where expert knowledge was no longer enough to manage the complex structures. This, in turn, led to the introduction of machine learning (ML) techniques in this rapidly developing and fascinating area. Here, we provide an overview of the main trends and EPF models as of 2022.
△ Less
Submitted 2 April, 2022;
originally announced April 2022.
-
Calibration window selection based on change-point detection for forecasting electricity prices
Authors:
Julia Nasiadka,
Weronika Nitka,
Rafał Weron
Abstract:
We employ a recently proposed change-point detection algorithm, the Narrowest-Over-Threshold (NOT) method, to select subperiods of past observations that are similar to the currently recorded values. Then, contrarily to the traditional time series approach in which the most recent $τ$ observations are taken as the calibration sample, we estimate autoregressive models only for data in these subperi…
▽ More
We employ a recently proposed change-point detection algorithm, the Narrowest-Over-Threshold (NOT) method, to select subperiods of past observations that are similar to the currently recorded values. Then, contrarily to the traditional time series approach in which the most recent $τ$ observations are taken as the calibration sample, we estimate autoregressive models only for data in these subperiods. We illustrate our approach using a challenging dataset - day-ahead electricity prices in the German EPEX SPOT market - and observe a significant improvement in forecasting accuracy compared to commonly used approaches, including the Autoregressive Hybrid Nearest Neighbors (ARHNN) method.
△ Less
Submitted 2 April, 2022;
originally announced April 2022.
-
Neural networks in day-ahead electricity price forecasting: Single vs. multiple outputs
Authors:
Grzegorz Marcjasz,
Jesus Lago,
Rafał Weron
Abstract:
Recent advancements in the fields of artificial intelligence and machine learning methods resulted in a significant increase of their popularity in the literature, including electricity price forecasting. Said methods cover a very broad spectrum, from decision trees, through random forests to various artificial neural network models and hybrid approaches. In electricity price forecasting, neural n…
▽ More
Recent advancements in the fields of artificial intelligence and machine learning methods resulted in a significant increase of their popularity in the literature, including electricity price forecasting. Said methods cover a very broad spectrum, from decision trees, through random forests to various artificial neural network models and hybrid approaches. In electricity price forecasting, neural networks are the most popular machine learning method as they provide a non-linear counterpart for well-tested linear regression models. Their application, however, is not straightforward, with multiple implementation factors to consider. One of such factors is the network's structure. This paper provides a comprehensive comparison of two most common structures when using the deep neural networks -- one that focuses on each hour of the day separately, and one that reflects the daily auction structure and models vectors of the prices. The results show a significant accuracy advantage of using the latter, confirmed on data from five distinct power exchanges.
△ Less
Submitted 18 August, 2020;
originally announced August 2020.
-
Forecasting day-ahead electricity prices: A review of state-of-the-art algorithms, best practices and an open-access benchmark
Authors:
Jesus Lago,
Grzegorz Marcjasz,
Bart De Schutter,
Rafał Weron
Abstract:
While the field of electricity price forecasting has benefited from plenty of contributions in the last two decades, it arguably lacks a rigorous approach to evaluating new predictive algorithms. The latter are often compared using unique, not publicly available datasets and across too short and limited to one market test samples. The proposed new methods are rarely benchmarked against well establ…
▽ More
While the field of electricity price forecasting has benefited from plenty of contributions in the last two decades, it arguably lacks a rigorous approach to evaluating new predictive algorithms. The latter are often compared using unique, not publicly available datasets and across too short and limited to one market test samples. The proposed new methods are rarely benchmarked against well established and well performing simpler models, the accuracy metrics are sometimes inadequate and testing the significance of differences in predictive performance is seldom conducted. Consequently, it is not clear which methods perform well nor what are the best practices when forecasting electricity prices. In this paper, we tackle these issues by performing a literature survey of state-of-the-art models, comparing state-of-the-art statistical and deep learning methods across multiple years and markets, and by putting forward a set of best practices. In addition, we make available the considered datasets, forecasts of the state-of-the-art models, and a specifically designed python toolbox, so that new algorithms can be rigorously evaluated in future studies.
△ Less
Submitted 21 December, 2020; v1 submitted 18 August, 2020;
originally announced August 2020.
-
Day-ahead electricity price forecasting with high-dimensional structures: Univariate vs. multivariate modeling frameworks
Authors:
Florian Ziel,
Rafal Weron
Abstract:
We conduct an extensive empirical study on short-term electricity price forecasting (EPF) to address the long-standing question if the optimal model structure for EPF is univariate or multivariate. We provide evidence that despite a minor edge in predictive performance overall, the multivariate modeling framework does not uniformly outperform the univariate one across all 12 considered datasets, s…
▽ More
We conduct an extensive empirical study on short-term electricity price forecasting (EPF) to address the long-standing question if the optimal model structure for EPF is univariate or multivariate. We provide evidence that despite a minor edge in predictive performance overall, the multivariate modeling framework does not uniformly outperform the univariate one across all 12 considered datasets, seasons of the year or hours of the day, and at times is outperformed by the latter. This is an indication that combining advanced structures or the corresponding forecasts from both modeling approaches can bring a further improvement in forecasting accuracy. We show that this indeed can be the case, even for a simple averaging scheme involving only two models. Finally, we also analyze variable selection for the best performing high-dimensional lasso-type models, thus provide guidelines to structuring better performing forecasting model designs.
△ Less
Submitted 17 May, 2018;
originally announced May 2018.
-
Black swans or dragon kings? A simple test for deviations from the power law
Authors:
Joanna Janczura,
Rafal Weron
Abstract:
We develop a simple test for deviations from power law tails, which is based on the asymptotic properties of the empirical distribution function. We use this test to answer the question whether great natural disasters, financial crashes or electricity price spikes should be classified as dragon kings or 'only' as black swans.
We develop a simple test for deviations from power law tails, which is based on the asymptotic properties of the empirical distribution function. We use this test to answer the question whether great natural disasters, financial crashes or electricity price spikes should be classified as dragon kings or 'only' as black swans.
△ Less
Submitted 17 February, 2011;
originally announced February 2011.
-
FX Smile in the Heston Model
Authors:
Agnieszka Janek,
Tino Kluge,
Rafal Weron,
Uwe Wystup
Abstract:
The Heston model stands out from the class of stochastic volatility (SV) models mainly for two reasons. Firstly, the process for the volatility is non-negative and mean-reverting, which is what we observe in the markets. Secondly, there exists a fast and easily implemented semi-analytical solution for European options. In this article we adapt the original work of Heston (1993) to a foreign exchan…
▽ More
The Heston model stands out from the class of stochastic volatility (SV) models mainly for two reasons. Firstly, the process for the volatility is non-negative and mean-reverting, which is what we observe in the markets. Secondly, there exists a fast and easily implemented semi-analytical solution for European options. In this article we adapt the original work of Heston (1993) to a foreign exchange (FX) setting. We discuss the computational aspects of using the semi-analytical formulas, performing Monte Carlo simulations, checking the Feller condition, and option pricing with FFT. In an empirical study we show that the smile of vanilla options can be reproduced by suitably calibrating three out of five model parameters.
△ Less
Submitted 8 October, 2010;
originally announced October 2010.
-
Outflow Dynamics in Modeling Oligopoly Markets: The Case of the Mobile Telecommunications Market in Poland
Authors:
Katarzyna Sznajd-Weron,
Rafał Weron,
Maja Włoszczowska
Abstract:
In this paper we introduce two models of opinion dynamics in oligopoly markets and apply them to a situation, where a new entrant challenges two incumbents of the same size. The models differ in the way the two forces influencing consumer choice -- (local) social interactions and (global) advertising -- interact. We study the general behavior of the models using the Mean Field Approach and Monte…
▽ More
In this paper we introduce two models of opinion dynamics in oligopoly markets and apply them to a situation, where a new entrant challenges two incumbents of the same size. The models differ in the way the two forces influencing consumer choice -- (local) social interactions and (global) advertising -- interact. We study the general behavior of the models using the Mean Field Approach and Monte Carlo simulations and calibrate the models to data from the Polish telecommunications market. For one of the models criticality is observed -- below a certain critical level of advertising the market approaches a lock-in situation, where one market leader dominates the market and all other brands disappear. Interestingly, for both models the best fits to real data are obtained for conformity level $p \in (0.3,0.4)$. This agrees very well with the conformity level found by Solomon Asch in his famous social experiment.
△ Less
Submitted 9 September, 2008;
originally announced September 2008.
-
How effective is advertising in duopoly markets?
Authors:
K. Sznajd-Weron,
R. Weron
Abstract:
A simple Ising spin model which can describe the mechanism of advertising in a duopoly market is proposed. In contrast to other agent-based models, the influence does not flow inward from the surrounding neighbors to the center site, but spreads outward from the center to the neighbors. The model thus describes the spread of opinions among customers. It is shown via standard Monte Carlo simulati…
▽ More
A simple Ising spin model which can describe the mechanism of advertising in a duopoly market is proposed. In contrast to other agent-based models, the influence does not flow inward from the surrounding neighbors to the center site, but spreads outward from the center to the neighbors. The model thus describes the spread of opinions among customers. It is shown via standard Monte Carlo simulations that very simple rules and inclusion of an external field -- an advertising campaign -- lead to phase transitions.
△ Less
Submitted 6 December, 2002; v1 submitted 4 November, 2002;
originally announced November 2002.
-
Measuring long-range dependence in electricity prices
Authors:
Rafal Weron
Abstract:
The price of electricity is far more volatile than that of other commodities normally noted for extreme volatility. The possibility of extreme price movements increases the risk of trading in electricity markets. However, underlying the process of price returns is a strong mean-reverting mechanism. We study this feature of electricity returns by means of Hurst R/S analysis, Detrended Fluctuation…
▽ More
The price of electricity is far more volatile than that of other commodities normally noted for extreme volatility. The possibility of extreme price movements increases the risk of trading in electricity markets. However, underlying the process of price returns is a strong mean-reverting mechanism. We study this feature of electricity returns by means of Hurst R/S analysis, Detrended Fluctuation Analysis and periodogram regression.
△ Less
Submitted 30 March, 2001;
originally announced March 2001.
-
A simple model of price formation
Authors:
K. Sznajd-Weron,
R. Weron
Abstract:
A simple Ising spin model which can describe the mechanism of price formation in financial markets is proposed. In contrast to other agent-based models, the influence does not flow inward from the surrounding neighbors to the center site, but spreads outward from the center to the neighbors. The model thus describes the spread of opinions among traders. It is shown via standard Monte Carlo simul…
▽ More
A simple Ising spin model which can describe the mechanism of price formation in financial markets is proposed. In contrast to other agent-based models, the influence does not flow inward from the surrounding neighbors to the center site, but spreads outward from the center to the neighbors. The model thus describes the spread of opinions among traders. It is shown via standard Monte Carlo simulations that very simple rules lead to dynamics that duplicate those of asset prices.
△ Less
Submitted 7 November, 2001; v1 submitted 30 December, 2000;
originally announced January 2001.
