-
Forecasting inflation using disaggregates and machine learning
Authors:
Gilberto Boaretto,
Marcelo C. Medeiros
Abstract:
This paper examines the effectiveness of several forecasting methods for predicting inflation, focusing on aggregating disaggregated forecasts - also known in the literature as the bottom-up approach. Taking the Brazilian case as an application, we consider different disaggregation levels for inflation and employ a range of traditional time series techniques as well as linear and nonlinear machine…
▽ More
This paper examines the effectiveness of several forecasting methods for predicting inflation, focusing on aggregating disaggregated forecasts - also known in the literature as the bottom-up approach. Taking the Brazilian case as an application, we consider different disaggregation levels for inflation and employ a range of traditional time series techniques as well as linear and nonlinear machine learning (ML) models to deal with a larger number of predictors. For many forecast horizons, the aggregation of disaggregated forecasts performs just as well survey-based expectations and models that generate forecasts using the aggregate directly. Overall, ML methods outperform traditional time series models in predictive accuracy, with outstanding performance in forecasting disaggregates. Our results reinforce the benefits of using models in a data-rich environment for inflation forecasting, including aggregating disaggregated forecasts from ML techniques, mainly during volatile periods. Starting from the COVID-19 pandemic, the random forest model based on both aggregate and disaggregated inflation achieves remarkable predictive performance at intermediate and longer horizons.
△ Less
Submitted 22 August, 2023;
originally announced August 2023.
-
Forecasting Large Realized Covariance Matrices: The Benefits of Factor Models and Shrinkage
Authors:
Rafael Alves,
Diego S. de Brito,
Marcelo C. Medeiros,
Ruy M. Ribeiro
Abstract:
We propose a model to forecast large realized covariance matrices of returns, applying it to the constituents of the S\&P 500 daily. To address the curse of dimensionality, we decompose the return covariance matrix using standard firm-level factors (e.g., size, value, and profitability) and use sectoral restrictions in the residual covariance matrix. This restricted model is then estimated using v…
▽ More
We propose a model to forecast large realized covariance matrices of returns, applying it to the constituents of the S\&P 500 daily. To address the curse of dimensionality, we decompose the return covariance matrix using standard firm-level factors (e.g., size, value, and profitability) and use sectoral restrictions in the residual covariance matrix. This restricted model is then estimated using vector heterogeneous autoregressive (VHAR) models with the least absolute shrinkage and selection operator (LASSO). Our methodology improves forecasting precision relative to standard benchmarks and leads to better estimates of minimum variance portfolios.
△ Less
Submitted 22 March, 2023;
originally announced March 2023.
-
Modeling and Forecasting Intraday Market Returns: a Machine Learning Approach
Authors:
Iuri H. Ferreira,
Marcelo C. Medeiros
Abstract:
In this paper we examine the relation between market returns and volatility measures through machine learning methods in a high-frequency environment. We implement a minute-by-minute rolling window intraday estimation method using two nonlinear models: Long-Short-Term Memory (LSTM) neural networks and Random Forests (RF). Our estimations show that the CBOE Volatility Index (VIX) is the strongest c…
▽ More
In this paper we examine the relation between market returns and volatility measures through machine learning methods in a high-frequency environment. We implement a minute-by-minute rolling window intraday estimation method using two nonlinear models: Long-Short-Term Memory (LSTM) neural networks and Random Forests (RF). Our estimations show that the CBOE Volatility Index (VIX) is the strongest candidate predictor for intraday market returns in our analysis, specially when implemented through the LSTM model. This model also improves significantly the performance of the lagged market return as predictive variable. Finally, intraday RF estimation outputs indicate that there is no performance improvement with this method, and it may even worsen the results in some cases.
△ Less
Submitted 30 December, 2021;
originally announced December 2021.
-
The Impacts of Mobility on Covid-19 Dynamics: Using Soft and Hard Data
Authors:
Leonardo Martins,
Marcelo C. Medeiros
Abstract:
This paper has the goal of evaluating how changes in mobility has affected the infection spread of Covid-19 throughout the 2020-2021 years. However, identifying a "clean" causal relation is not an easy task due to a high number of non-observable (behavioral) effects. We suggest the usage of Google Trends and News-based indexes as controls for some of these behavioral effects and we find that a 1\%…
▽ More
This paper has the goal of evaluating how changes in mobility has affected the infection spread of Covid-19 throughout the 2020-2021 years. However, identifying a "clean" causal relation is not an easy task due to a high number of non-observable (behavioral) effects. We suggest the usage of Google Trends and News-based indexes as controls for some of these behavioral effects and we find that a 1\% increase in residential mobility (i.e. a reduction in overall mobility) have significant impacts for reducing both Covid-19 cases (at least 3.02\% on a one-month horizon) and deaths (at least 2.43\% at the two-weeks horizon) over the 2020-2021 sample. We also evaluate the effects of mobility on Covid-19 spread on the restricted sample (only 2020) where vaccines were not available. The results of diminishing mobility over cases and deaths on the restricted sample are still observable (with similar magnitudes in terms of residential mobility) and cumulative higher, as the effects of restricting workplace mobility turns to be also significant: a 1\% decrease in workplace mobility diminishes cases around 1\% and deaths around 2\%.
△ Less
Submitted 1 October, 2021;
originally announced October 2021.
-
The Proper Use of Google Trends in Forecasting Models
Authors:
Marcelo C. Medeiros,
Henrique F. Pires
Abstract:
It is widely known that Google Trends have become one of the most popular free tools used by forecasters both in academics and in the private and public sectors. There are many papers, from several different fields, concluding that Google Trends improve forecasts' accuracy. However, what seems to be widely unknown, is that each sample of Google search data is different from the other, even if you…
▽ More
It is widely known that Google Trends have become one of the most popular free tools used by forecasters both in academics and in the private and public sectors. There are many papers, from several different fields, concluding that Google Trends improve forecasts' accuracy. However, what seems to be widely unknown, is that each sample of Google search data is different from the other, even if you set the same search term, data and location. This means that it is possible to find arbitrary conclusions merely by chance. This paper aims to show why and when it can become a problem and how to overcome this obstacle.
△ Less
Submitted 10 April, 2021; v1 submitted 7 April, 2021;
originally announced April 2021.
-
Bridging factor and sparse models
Authors:
Jianqing Fan,
Ricardo Masini,
Marcelo C. Medeiros
Abstract:
Factor and sparse models are two widely used methods to impose a low-dimensional structure in high-dimensions. However, they are seemingly mutually exclusive. We propose a lifting method that combines the merits of these two models in a supervised learning methodology that allows for efficiently exploring all the information in high-dimensional datasets. The method is based on a flexible model for…
▽ More
Factor and sparse models are two widely used methods to impose a low-dimensional structure in high-dimensions. However, they are seemingly mutually exclusive. We propose a lifting method that combines the merits of these two models in a supervised learning methodology that allows for efficiently exploring all the information in high-dimensional datasets. The method is based on a flexible model for high-dimensional panel data, called factor-augmented regression model with observable and/or latent common factors, as well as idiosyncratic components. This model not only includes both principal component regression and sparse regression as specific models but also significantly weakens the cross-sectional dependence and facilitates model selection and interpretability. The method consists of several steps and a novel test for (partial) covariance structure in high dimensions to infer the remaining cross-section dependence at each step. We develop the theory for the model and demonstrate the validity of the multiplier bootstrap for testing a high-dimensional (partial) covariance structure. The theory is supported by a simulation study and applications.
△ Less
Submitted 3 September, 2022; v1 submitted 22 February, 2021;
originally announced February 2021.
-
Machine Learning Advances for Time Series Forecasting
Authors:
Ricardo P. Masini,
Marcelo C. Medeiros,
Eduardo F. Mendes
Abstract:
In this paper we survey the most recent advances in supervised machine learning and high-dimensional models for time series forecasting. We consider both linear and nonlinear alternatives. Among the linear methods we pay special attention to penalized regressions and ensemble of models. The nonlinear methods considered in the paper include shallow and deep neural networks, in their feed-forward an…
▽ More
In this paper we survey the most recent advances in supervised machine learning and high-dimensional models for time series forecasting. We consider both linear and nonlinear alternatives. Among the linear methods we pay special attention to penalized regressions and ensemble of models. The nonlinear methods considered in the paper include shallow and deep neural networks, in their feed-forward and recurrent versions, and tree-based methods, such as random forests and boosted trees. We also consider ensemble and hybrid models by combining ingredients from different alternatives. Tests for superior predictive ability are briefly reviewed. Finally, we discuss application of machine learning in economics and finance and provide an illustration with high-frequency financial data.
△ Less
Submitted 9 April, 2021; v1 submitted 23 December, 2020;
originally announced December 2020.
-
Do We Exploit all Information for Counterfactual Analysis? Benefits of Factor Models and Idiosyncratic Correction
Authors:
Jianqing Fan,
Ricardo P. Masini,
Marcelo C. Medeiros
Abstract:
Optimal pricing, i.e., determining the price level that maximizes profit or revenue of a given product, is a vital task for the retail industry. To select such a quantity, one needs first to estimate the price elasticity from the product demand. Regression methods usually fail to recover such elasticities due to confounding effects and price endogeneity. Therefore, randomized experiments are typic…
▽ More
Optimal pricing, i.e., determining the price level that maximizes profit or revenue of a given product, is a vital task for the retail industry. To select such a quantity, one needs first to estimate the price elasticity from the product demand. Regression methods usually fail to recover such elasticities due to confounding effects and price endogeneity. Therefore, randomized experiments are typically required. However, elasticities can be highly heterogeneous depending on the location of stores, for example. As the randomization frequently occurs at the municipal level, standard difference-in-differences methods may also fail. Possible solutions are based on methodologies to measure the effects of treatments on a single (or just a few) treated unit(s) based on counterfactuals constructed from artificial controls. For example, for each city in the treatment group, a counterfactual may be constructed from the untreated locations. In this paper, we apply a novel high-dimensional statistical method to measure the effects of price changes on daily sales from a major retailer in Brazil. The proposed methodology combines principal components (factors) and sparse regressions, resulting in a method called Factor-Adjusted Regularized Method for Treatment evaluation (\texttt{FarmTreat}). The data consist of daily sales and prices of five different products over more than 400 municipalities. The products considered belong to the \emph{sweet and candies} category and experiments have been conducted over the years of 2016 and 2017. Our results confirm the hypothesis of a high degree of heterogeneity yielding very different pricing strategies over distinct municipalities.
△ Less
Submitted 10 January, 2022; v1 submitted 8 November, 2020;
originally announced November 2020.
-
Online Action Learning in High Dimensions: A Conservative Perspective
Authors:
Claudio Cardoso Flores,
Marcelo Cunha Medeiros
Abstract:
Sequential learning problems are common in several fields of research and practical applications. Examples include dynamic pricing and assortment, design of auctions and incentives and permeate a large number of sequential treatment experiments. In this paper, we extend one of the most popular learning solutions, the $ε_t$-greedy heuristics, to high-dimensional contexts considering a conservative…
▽ More
Sequential learning problems are common in several fields of research and practical applications. Examples include dynamic pricing and assortment, design of auctions and incentives and permeate a large number of sequential treatment experiments. In this paper, we extend one of the most popular learning solutions, the $ε_t$-greedy heuristics, to high-dimensional contexts considering a conservative directive. We do this by allocating part of the time the original rule uses to adopt completely new actions to a more focused search in a restrictive set of promising actions. The resulting rule might be useful for practical applications that still values surprises, although at a decreasing rate, while also has restrictions on the adoption of unusual actions. With high probability, we find reasonable bounds for the cumulative regret of a conservative high-dimensional decaying $ε_t$-greedy rule. Also, we provide a lower bound for the cardinality of the set of viable actions that implies in an improved regret bound for the conservative version when compared to its non-conservative counterpart. Additionally, we show that end-users have sufficient flexibility when establishing how much safety they want, since it can be tuned without impacting theoretical properties. We illustrate our proposal both in a simulation exercise and using a real dataset.
△ Less
Submitted 23 March, 2024; v1 submitted 29 September, 2020;
originally announced September 2020.
-
Lockdown effects in US states: an artificial counterfactual approach
Authors:
Carlos B. Carneiro,
Iúri H. Ferreira,
Marcelo C. Medeiros,
Henrique F. Pires,
Eduardo Zilberman
Abstract:
We adopt an artificial counterfactual approach to assess the impact of lockdowns on the short-run evolution of the number of cases and deaths in some US states. To do so, we explore the different timing in which US states adopted lockdown policies, and divide them among treated and control groups. For each treated state, we construct an artificial counterfactual. On average, and in the very short-…
▽ More
We adopt an artificial counterfactual approach to assess the impact of lockdowns on the short-run evolution of the number of cases and deaths in some US states. To do so, we explore the different timing in which US states adopted lockdown policies, and divide them among treated and control groups. For each treated state, we construct an artificial counterfactual. On average, and in the very short-run, the counterfactual accumulated number of cases would be two times larger if lockdown policies were not implemented.
△ Less
Submitted 8 February, 2021; v1 submitted 28 September, 2020;
originally announced September 2020.
-
Regularized Estimation of High-Dimensional Vector AutoRegressions with Weakly Dependent Innovations
Authors:
Ricardo P. Masini,
Marcelo C. Medeiros,
Eduardo F. Mendes
Abstract:
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive models with heavy tailed, weakly dependent innovations with virtually no assumption o…
▽ More
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive models with heavy tailed, weakly dependent innovations with virtually no assumption on the conditional heteroskedasticity. In contrast to current literature, our innovation process satisfy an $L^1$ mixingale type condition on the centered conditional covariance matrices. This condition covers $L^1$-NED sequences and strong ($α$-) mixing sequences as particular examples.
△ Less
Submitted 11 June, 2021; v1 submitted 18 December, 2019;
originally announced December 2019.
