-
Forecasting Italian daily electricity generation disaggregated by geographical zones and energy sources using coherent forecast combination
Authors:
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
A novel approach is applied for improving forecast accuracy and achieving coherence in forecasting the Italian daily energy generation time series. In hierarchical frameworks such as national energy generation disaggregated by geographical zones and energy sources, independently generated base forecasts often result in inconsistencies across the constraints. We deal with this issue through a coher…
▽ More
A novel approach is applied for improving forecast accuracy and achieving coherence in forecasting the Italian daily energy generation time series. In hierarchical frameworks such as national energy generation disaggregated by geographical zones and energy sources, independently generated base forecasts often result in inconsistencies across the constraints. We deal with this issue through a coherent balanced multi-task forecast combination approach, which combines unbiased forecasts from multiple experts while ensuring coherence. Applied to the daily Italian electricity generation data, our method shows superior accuracy compared to single-task base and combined forecasts, and a state-of-the-art single-expert reconciliation technique, demonstrating to be an effective approach to forecasting linearly constrained multiple time series.
△ Less
Submitted 17 February, 2025;
originally announced February 2025.
-
Energy load forecasting using Terna public data: a free lunch multi-task combination approach
Authors:
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
We propose a quick-and-simple procedure to augment the accuracy of 15-minutes Italian load forecasts disaggregated by bidding zones published by Terna, the operator of the Italian electricity system. We show that a stacked-regression multi-task combination approach using Terna and daily random walk naive forecasts, is able to produce significantly more accurate forecasts immediately after Terna pu…
▽ More
We propose a quick-and-simple procedure to augment the accuracy of 15-minutes Italian load forecasts disaggregated by bidding zones published by Terna, the operator of the Italian electricity system. We show that a stacked-regression multi-task combination approach using Terna and daily random walk naive forecasts, is able to produce significantly more accurate forecasts immediately after Terna publishes on its data portal the energy load measurements for the previous day, and the forecasts for the current day.
△ Less
Submitted 17 February, 2025;
originally announced February 2025.
-
Balancing Accuracy and Costs in Cross-Temporal Hierarchies: Investigating Decision-Based and Validation-Based Reconciliation
Authors:
Mahdi Abolghasemi,
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
Wind power forecasting is essential for managing daily operations at wind farms and enabling market operators to manage power uncertainty effectively in demand planning. This paper explores advanced cross-temporal forecasting models and their potential to enhance forecasting accuracy. First, we propose a novel approach that leverages validation errors, rather than traditional in-sample errors, for…
▽ More
Wind power forecasting is essential for managing daily operations at wind farms and enabling market operators to manage power uncertainty effectively in demand planning. This paper explores advanced cross-temporal forecasting models and their potential to enhance forecasting accuracy. First, we propose a novel approach that leverages validation errors, rather than traditional in-sample errors, for covariance matrix estimation and forecast reconciliation. Second, we introduce decision-based aggregation levels for forecasting and reconciliation where certain horizons are based on the required decisions in practice. Third, we evaluate the forecasting performance of the models not only on their ability to minimize errors but also on their effectiveness in reducing decision costs, such as penalties in ancillary services. Our results show that statistical-based hierarchies tend to adopt less conservative forecasts and reduce revenue losses. On the other hand, decision-based reconciliation offers a more balanced compromise between accuracy and decision cost, making them attractive for practical use.
△ Less
Submitted 15 December, 2024;
originally announced December 2024.
-
Coherent forecast combination for linearly constrained multiple time series
Authors:
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
Linearly constrained multiple time series may be encountered in many practical contexts, such as the National Accounts (e.g., GDP disaggregated by Income, Expenditure and Output), and multilevel frameworks where the variables are organized according to hierarchies or groupings, like the total energy consumption of a country disaggregated by region and energy sources. In these cases, when multiple…
▽ More
Linearly constrained multiple time series may be encountered in many practical contexts, such as the National Accounts (e.g., GDP disaggregated by Income, Expenditure and Output), and multilevel frameworks where the variables are organized according to hierarchies or groupings, like the total energy consumption of a country disaggregated by region and energy sources. In these cases, when multiple incoherent base forecasts for each individual variable are available, a forecast combination-and-reconciliation approach, that we call coherent forecast combination, may be used to improve the accuracy of the base forecasts and achieve coherence in the final result. In this paper, we develop an optimization-based technique that combines multiple unbiased base forecasts while assuring the constraints valid for the series. We present closed form expressions for the coherent combined forecast vector and its error covariance matrix in the general case where a different number of forecasts is available for each variable. We also discuss practical issues related to the covariance matrix that is part of the optimal solution. Through simulations and a forecasting experiment on the daily Australian electricity generation hierarchical time series, we show that the proposed methodology, in addition to adhering to sound statistical principles, may yield in significant improvement on base forecasts, single-task combination and single-expert reconciliation approaches as well.
△ Less
Submitted 4 December, 2024;
originally announced December 2024.
-
Insights into regression-based cross-temporal forecast reconciliation
Authors:
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
Cross-temporal forecast reconciliation aims to ensure consistency across forecasts made at different temporal and cross-sectional levels. We explore the relationships between sequential, iterative, and optimal combination approaches, and discuss the conditions under which a sequential reconciliation approach (either first-cross-sectional-then-temporal, or first-temporal-then-cross-sectional) is eq…
▽ More
Cross-temporal forecast reconciliation aims to ensure consistency across forecasts made at different temporal and cross-sectional levels. We explore the relationships between sequential, iterative, and optimal combination approaches, and discuss the conditions under which a sequential reconciliation approach (either first-cross-sectional-then-temporal, or first-temporal-then-cross-sectional) is equivalent to a fully (i.e., cross-temporally) coherent iterative heuristic. Furthermore, we show that for specific patterns of the error covariance matrix in the regression model on which the optimal combination approach grounds, iterative reconciliation naturally converges to the optimal combination solution, regardless the order of application of the uni-dimensional cross-sectional and temporal reconciliation approaches. Theoretical and empirical properties of the proposed approaches are investigated through a forecasting experiment using a dataset of hourly photovoltaic power generation. The study presents a comprehensive framework for understanding and enhancing cross-temporal forecast reconciliation, considering both forecast accuracy and the often overlooked computational aspects, showing that significant improvement can be achieved in terms of memory space and computation time, two particularly important aspects in the high-dimensional contexts that usually arise in cross-temporal forecast reconciliation.
△ Less
Submitted 25 October, 2024;
originally announced October 2024.
-
Exploiting Intraday Decompositions in Realized Volatility Forecasting: A Forecast Reconciliation Approach
Authors:
Massimiliano Caporin,
Tommaso Di Fonzo,
Daniele Girolimetto
Abstract:
We address the construction of Realized Variance (RV) forecasts by exploiting the hierarchical structure implicit in available decompositions of RV. By using data referred to the Dow Jones Industrial Average Index and to its constituents we show that exploiting the informative content of hierarchies improves the forecast accuracy. Forecasting performance is evaluated out-of-sample based on the emp…
▽ More
We address the construction of Realized Variance (RV) forecasts by exploiting the hierarchical structure implicit in available decompositions of RV. By using data referred to the Dow Jones Industrial Average Index and to its constituents we show that exploiting the informative content of hierarchies improves the forecast accuracy. Forecasting performance is evaluated out-of-sample based on the empirical MSE and QLIKE criteria as well as using the Model Confidence Set approach.
△ Less
Submitted 5 June, 2023;
originally announced June 2023.
-
Point and probabilistic forecast reconciliation for general linearly constrained multiple time series
Authors:
Daniele Girolimetto,
Tommaso Di Fonzo
Abstract:
Forecast reconciliation is the post-forecasting process aimed to revise a set of incoherent base forecasts into coherent forecasts in line with given data structures. Most of the point and probabilistic regression-based forecast reconciliation results ground on the so called "structural representation" and on the related unconstrained generalized least squares reconciliation formula. However, the…
▽ More
Forecast reconciliation is the post-forecasting process aimed to revise a set of incoherent base forecasts into coherent forecasts in line with given data structures. Most of the point and probabilistic regression-based forecast reconciliation results ground on the so called "structural representation" and on the related unconstrained generalized least squares reconciliation formula. However, the structural representation naturally applies to genuine hierarchical/grouped time series, where the top- and bottom-level variables are uniquely identified. When a general linearly constrained multiple time series is considered, the forecast reconciliation is naturally expressed according to a projection approach. While it is well known that the classic structural reconciliation formula is equivalent to its projection approach counterpart, so far it is not completely understood if and how a structural-like reconciliation formula may be derived for a general linearly constrained multiple time series. Such an expression would permit to extend reconciliation definitions, theorems and results in a straightforward manner. In this paper, we show that for general linearly constrained multiple time series it is possible to express the reconciliation formula according to a "structural-like" approach that keeps distinct free and constrained, instead of bottom and upper (aggregated), variables, establish the probabilistic forecast reconciliation framework, and apply these findings to obtain fully reconciled point and probabilistic forecasts for the aggregates of the Australian GDP from income and expenditure sides, and for the European Area GDP disaggregated by income, expenditure and output sides and by 19 countries.
△ Less
Submitted 22 December, 2023; v1 submitted 9 May, 2023;
originally announced May 2023.
-
Cross-temporal probabilistic forecast reconciliation: Methodological and practical issues
Authors:
Daniele Girolimetto,
George Athanasopoulos,
Tommaso Di Fonzo,
Rob J Hyndman
Abstract:
Forecast reconciliation is a post-forecasting process that involves transforming a set of incoherent forecasts into coherent forecasts which satisfy a given set of linear constraints for a multivariate time series. In this paper we extend the current state-of-the-art cross-sectional probabilistic forecast reconciliation approach to encompass a cross-temporal framework, where temporal constraints a…
▽ More
Forecast reconciliation is a post-forecasting process that involves transforming a set of incoherent forecasts into coherent forecasts which satisfy a given set of linear constraints for a multivariate time series. In this paper we extend the current state-of-the-art cross-sectional probabilistic forecast reconciliation approach to encompass a cross-temporal framework, where temporal constraints are also applied. Our proposed methodology employs both parametric Gaussian and non-parametric bootstrap approaches to draw samples from an incoherent cross-temporal distribution. To improve the estimation of the forecast error covariance matrix, we propose using multi-step residuals, especially in the time dimension where the usual one-step residuals fail. To address high-dimensionality issues, we present four alternatives for the covariance matrix, where we exploit the two-fold nature (cross-sectional and temporal) of the cross-temporal structure, and introduce the idea of overlapping residuals. We assess the effectiveness of the proposed cross-temporal reconciliation approaches through a simulation study that investigates their theoretical and empirical properties and two forecasting experiments, using the Australian GDP and the Australian Tourism Demand datasets. For both applications, the optimal cross-temporal reconciliation approaches significantly outperform the incoherent base forecasts in terms of the Continuous Ranked Probability Score and the Energy Score. Overall, the results highlight the potential of the proposed methods to improve the accuracy of probabilistic forecasts and to address the challenge of integrating disparate scenarios while coherently taking into account short-term operational, medium-term tactical, and long-term strategic planning.
△ Less
Submitted 27 October, 2023; v1 submitted 30 March, 2023;
originally announced March 2023.
-
Enhancements in cross-temporal forecast reconciliation, with an application to solar irradiance forecasts
Authors:
Tommaso Di Fonzo,
Daniele Girolimetto
Abstract:
In recent works by Yang et al. (2017a,b), and Yagli et al. (2019), geographical, temporal, and sequential deterministic reconciliation of hierarchical photovoltaic (PV) power generation have been considered for a simulated PV dataset in California. In the first two cases, the reconciliations are carried out in spatial and temporal domains separately. To further improve forecasting accuracy, in the…
▽ More
In recent works by Yang et al. (2017a,b), and Yagli et al. (2019), geographical, temporal, and sequential deterministic reconciliation of hierarchical photovoltaic (PV) power generation have been considered for a simulated PV dataset in California. In the first two cases, the reconciliations are carried out in spatial and temporal domains separately. To further improve forecasting accuracy, in the third case these two reconciliation approaches are sequentially applied. During the replication of the forecasting experiment, some issues emerged about non-negativity and coherence (in space and/or in time) of the sequentially reconciled forecasts. Furthermore, while the accuracy improvement of the considered approaches over the benchmark persistence forecasts is clearly visible at any data granularity, we argue that an even better performance may be obtained by a thorough exploitation of cross-temporal hierarchies. In this paper the cross-temporal point forecast reconciliation approach is applied to generate non-negative, fully coherent (both in space and time) forecasts. In particular, some relationships between two-step, iterative and simultaneous cross-temporal reconciliation procedures are for the first time established, non-negativity issues of the final reconciled forecasts are correctly dealt with in a simple way, and the most recent cross-temporal reconciliation approaches are adopted. The normalised Root Mean Square Error is used to measure forecasting accuracy, and a statistical multiple comparison procedure is performed to rank the approaches. Besides assuring full coherence, and non-negativity of the reconciled forecasts, the results show that for the considered dataset, cross-temporal forecast reconciliation significantly improves on the sequential procedures proposed by Yagli et al. (2019), at any cross-sectional level of the hierarchy and for any temporal granularity.
△ Less
Submitted 15 September, 2022;
originally announced September 2022.
-
Forecast combination based forecast reconciliation: insights and extensions
Authors:
Tommaso Di Fonzo,
Daniele Girolimetto
Abstract:
In a recent paper, while elucidating the links between forecast combination and cross-sectional forecast reconciliation, Hollyman et al. (2021) have proposed a forecast combination-based approach to the reconciliation of a simple hierarchy. A new Level Conditional Coherent (LCC) point forecast reconciliation procedure was developed, and it was shown that the simple average of a set of LCC, and bot…
▽ More
In a recent paper, while elucidating the links between forecast combination and cross-sectional forecast reconciliation, Hollyman et al. (2021) have proposed a forecast combination-based approach to the reconciliation of a simple hierarchy. A new Level Conditional Coherent (LCC) point forecast reconciliation procedure was developed, and it was shown that the simple average of a set of LCC, and bottom-up reconciled forecasts (called Combined Conditional Coherent, CCC) results in good performance as compared to those obtained through the state-of-the-art cross-sectional reconciliation procedures. In this paper, we build upon and extend this proposal along some new directions. (1) We shed light on the nature and the mathematical derivation of the LCC reconciliation formula, showing that it is the result of an exogenously linearly constrained minimization of a quadratic loss function in the differences between the target and the base forecasts with a diagonal associated matrix. (2) Endogenous constraints may be considered as well, resulting in level conditional reconciled forecasts of all the involved series, where both the upper and the bottom time series are coherently revised. (3) As the LCC procedure does not guarantee the non-negativity of the reconciled forecasts, we argue that - when non-negativity is a natural attribute of the variables to be forecast - its interpretation as an unbiased top-down reconciliation procedure leaves room for some doubts. (4) The new procedures are used in a forecasting experiment on the classical Australian Tourism Demand (Visitor Nights) dataset. Due to the crucial role played by the (possibly different) models used to compute the base forecasts, we re-interpret the CCC reconciliation of Hollyman et al. (2021) as a forecast pooling approach, showing that accuracy improvement may be gained by adopting a simple forecast averaging strategy.
△ Less
Submitted 10 June, 2021;
originally announced June 2021.
-
Cross-temporal forecast reconciliation: Optimal combination method and heuristic alternatives
Authors:
Tommaso Di Fonzo,
Daniele Girolimetto
Abstract:
Forecast reconciliation is a post-forecasting process aimed to improve the quality of the base forecasts for a system of hierarchical/grouped time series (Hyndman et al., 2011). Contemporaneous (cross-sectional) and temporal hierarchies have been considered in the literature, but - except for Kourentzes and Athanasopoulos (2019) - generally these two features have not been fully considered togethe…
▽ More
Forecast reconciliation is a post-forecasting process aimed to improve the quality of the base forecasts for a system of hierarchical/grouped time series (Hyndman et al., 2011). Contemporaneous (cross-sectional) and temporal hierarchies have been considered in the literature, but - except for Kourentzes and Athanasopoulos (2019) - generally these two features have not been fully considered together. Adopting a notation able to simultaneously deal with both forecast reconciliation dimensions, the paper shows two new results: (i) an iterative cross-temporal forecast reconciliation procedure which extends, and overcomes some weaknesses of, the two-step procedure by Kourentzes and Athanasopoulos (2019), and (ii) the closed-form expression of the optimal (in least squares sense) point forecasts which fulfill both contemporaneous and temporal constraints. The feasibility of the proposed procedures, along with first evaluations of their performance as compared to the most performing `single dimension' (either cross-sectional or temporal) forecast reconciliation procedures, is studied through a forecasting experiment on the 95 quarterly time series of the Australian GDP from Income and Expenditure sides considered by Athanasopoulos et al. (2019).
△ Less
Submitted 19 October, 2020; v1 submitted 15 June, 2020;
originally announced June 2020.
-
Fully reconciled GDP forecasts from Income and Expenditure sides
Authors:
Luisa Bisaglia,
Tommaso Di Fonzo,
Daniele Girolimetto
Abstract:
We propose a complete reconciliation procedure, resulting in a 'one number forecast' of the GDP figure, coherent with both Income and Expenditure sides' forecasted series, and evaluate its performance on the Australian quarterly GDP series, as compared to the original proposal by Athanasopoulos et al. (2019).
We propose a complete reconciliation procedure, resulting in a 'one number forecast' of the GDP figure, coherent with both Income and Expenditure sides' forecasted series, and evaluate its performance on the Australian quarterly GDP series, as compared to the original proposal by Athanasopoulos et al. (2019).
△ Less
Submitted 10 May, 2020; v1 submitted 8 April, 2020;
originally announced April 2020.
