-
Efficient Data-Driven Leverage Score Sampling Algorithm for the Minimum Volume Covering Ellipsoid Problem in Big Data
Authors:
Elizabeth Harris,
Ali Eshragh,
Bishnu Lamichhane,
Jordan Shaw-Carmody,
Elizabeth Stojanovski
Abstract:
The Minimum Volume Covering Ellipsoid (MVCE) problem, characterised by $n$ observations in $d$ dimensions where $n \gg d$, can be computationally very expensive in the big data regime. We apply methods from randomised numerical linear algebra to develop a data-driven leverage score sampling algorithm for solving MVCE, and establish theoretical error bounds and a convergence guarantee. Assuming the…
▽ More
The Minimum Volume Covering Ellipsoid (MVCE) problem, characterised by $n$ observations in $d$ dimensions where $n \gg d$, can be computationally very expensive in the big data regime. We apply methods from randomised numerical linear algebra to develop a data-driven leverage score sampling algorithm for solving MVCE, and establish theoretical error bounds and a convergence guarantee. Assuming the leverage scores follow a power law decay, we show that the computational complexity of computing the approximation for MVCE is reduced from $\mathcal{O}(nd^2)$ to $\mathcal{O}(nd + \text{poly}(d))$, which is a significant improvement in big data problems. Numerical experiments demonstrate the efficacy of our new algorithm, showing that it substantially reduces computation time and yields near-optimal solutions.
△ Less
Submitted 5 November, 2024;
originally announced November 2024.
-
SALSA: Sequential Approximate Leverage-Score Algorithm with Application in Analyzing Big Time Series Data
Authors:
Ali Eshragh,
Luke Yerbury,
Asef Nazari,
Fred Roosta,
Michael W. Mahoney
Abstract:
We develop a new efficient sequential approximate leverage score algorithm, SALSA, using methods from randomized numerical linear algebra (RandNLA) for large matrices. We demonstrate that, with high probability, the accuracy of SALSA's approximations is within $(1 + O({\varepsilon}))$ of the true leverage scores. In addition, we show that the theoretical computational complexity and numerical accu…
▽ More
We develop a new efficient sequential approximate leverage score algorithm, SALSA, using methods from randomized numerical linear algebra (RandNLA) for large matrices. We demonstrate that, with high probability, the accuracy of SALSA's approximations is within $(1 + O({\varepsilon}))$ of the true leverage scores. In addition, we show that the theoretical computational complexity and numerical accuracy of SALSA surpass existing approximations. These theoretical results are subsequently utilized to develop an efficient algorithm, named LSARMA, for fitting an appropriate ARMA model to large-scale time series data. Our proposed algorithm is, with high probability, guaranteed to find the maximum likelihood estimates of the parameters for the true underlying ARMA model. Furthermore, it has a worst-case running time that significantly improves those of the state-of-the-art alternatives in big data regimes. Empirical results on large-scale data strongly support these theoretical results and underscore the efficacy of our new approach.
△ Less
Submitted 29 December, 2023;
originally announced January 2024.
-
A Hybrid Statistical-Machine Learning Approach for Analysing Online Customer Behavior: An Empirical Study
Authors:
Saed Alizamir,
Kasun Bandara,
Ali Eshragh,
Foaad Iravani
Abstract:
We apply classical statistical methods in conjunction with the state-of-the-art machine learning techniques to develop a hybrid interpretable model to analyse 454,897 online customers' behavior for a particular product category at the largest online retailer in China, that is JD. While most mere machine learning methods are plagued by the lack of interpretability in practice, our novel hybrid appr…
▽ More
We apply classical statistical methods in conjunction with the state-of-the-art machine learning techniques to develop a hybrid interpretable model to analyse 454,897 online customers' behavior for a particular product category at the largest online retailer in China, that is JD. While most mere machine learning methods are plagued by the lack of interpretability in practice, our novel hybrid approach will address this practical issue by generating explainable output. This analysis involves identifying what features and characteristics have the most significant impact on customers' purchase behavior, thereby enabling us to predict future sales with a high level of accuracy, and identify the most impactful variables. Our results reveal that customers' product choice is insensitive to the promised delivery time, but this factor significantly impacts customers' order quantity. We also show that the effectiveness of various discounting methods depends on the specific product and the discount size. We identify product classes for which certain discounting approaches are more effective and provide recommendations on better use of different discounting tools. Customers' choice behavior across different product classes is mostly driven by price, and to a lesser extent, by customer demographics. The former finding asks for exercising care in deciding when and how much discount should be offered, whereas the latter identifies opportunities for personalized ads and targeted marketing. Further, to curb customers' batch ordering behavior and avoid the undesirable Bullwhip effect, JD should improve its logistics to ensure faster delivery of orders.
△ Less
Submitted 1 December, 2022;
originally announced December 2022.
-
Toeplitz Least Squares Problems, Fast Algorithms and Big Data
Authors:
Ali Eshragh,
Oliver Di Pietro,
Michael A. Saunders
Abstract:
In time series analysis, when fitting an autoregressive model, one must solve a Toeplitz ordinary least squares problem numerous times to find an appropriate model, which can severely affect computational times with large data sets. Two recent algorithms (LSAR and Repeated Halving) have applied randomized numerical linear algebra (RandNLA) techniques to fitting an autoregressive model to big time-…
▽ More
In time series analysis, when fitting an autoregressive model, one must solve a Toeplitz ordinary least squares problem numerous times to find an appropriate model, which can severely affect computational times with large data sets. Two recent algorithms (LSAR and Repeated Halving) have applied randomized numerical linear algebra (RandNLA) techniques to fitting an autoregressive model to big time-series data. We investigate and compare the quality of these two approximation algorithms on large-scale synthetic and real-world data. While both algorithms display comparable results for synthetic datasets, the LSAR algorithm appears to be more robust when applied to real-world time series data. We conclude that RandNLA is effective in the context of big-data time series.
△ Less
Submitted 24 December, 2021;
originally announced December 2021.
-
Rollage: Efficient Rolling Average Algorithm to Estimate ARMA Models for Big Time Series Data
Authors:
Ali Eshragh,
Glen Livingston,
Thomas McCarthy McCann,
Luke Yerbury
Abstract:
We develop a new efficient algorithm for the analysis of large-scale time series data. We firstly define rolling averages, derive their analytical properties, and establish their asymptotic distribution. These theoretical results are subsequently exploited to develop an efficient algorithm, called Rollage, for fitting an appropriate AR model to big time series data. When used in conjunction with t…
▽ More
We develop a new efficient algorithm for the analysis of large-scale time series data. We firstly define rolling averages, derive their analytical properties, and establish their asymptotic distribution. These theoretical results are subsequently exploited to develop an efficient algorithm, called Rollage, for fitting an appropriate AR model to big time series data. When used in conjunction with the Durbin's algorithm, we show that the Rollage algorithm can be used as a criterion to optimally fit ARMA models to big time series data. Empirical experiments on large-scale synthetic time series data support the theoretical results and reveal the efficacy of this new approach, especially when compared to existing methodology.
△ Less
Submitted 23 December, 2022; v1 submitted 16 March, 2021;
originally announced March 2021.
-
Modeling the Dynamics of the COVID-19 Population in Australia: A Probabilistic Analysis
Authors:
Ali Eshragh,
Saed Alizamir,
Peter Howley,
Elizabeth Stojanovski
Abstract:
The novel Corona Virus COVID-19 arrived on Australian shores around 25 January 2020. This paper presents a novel method of dynamically modeling and forecasting the COVID-19 pandemic in Australia with a high degree of accuracy and in a timely manner using limited data; a valuable resource that can be used to guide government decision-making on societal restrictions on a daily and/or weekly basis. T…
▽ More
The novel Corona Virus COVID-19 arrived on Australian shores around 25 January 2020. This paper presents a novel method of dynamically modeling and forecasting the COVID-19 pandemic in Australia with a high degree of accuracy and in a timely manner using limited data; a valuable resource that can be used to guide government decision-making on societal restrictions on a daily and/or weekly basis. The "partially-observable stochastic process" used in this study predicts not only the future actual values with extremely low error, but also the percentage of unobserved COVID-19 cases in the population. The model can further assist policy makers to assess the effectiveness of several possible alternative scenarios in their decision-making processes.
△ Less
Submitted 25 May, 2020;
originally announced May 2020.
-
LSAR: Efficient Leverage Score Sampling Algorithm for the Analysis of Big Time Series Data
Authors:
Ali Eshragh,
Fred Roosta,
Asef Nazari,
Michael W. Mahoney
Abstract:
We apply methods from randomized numerical linear algebra (RandNLA) to develop improved algorithms for the analysis of large-scale time series data. We first develop a new fast algorithm to estimate the leverage scores of an autoregressive (AR) model in big data regimes. We show that the accuracy of approximations lies within $(1+\bigO{\varepsilon})$ of the true leverage scores with high probabili…
▽ More
We apply methods from randomized numerical linear algebra (RandNLA) to develop improved algorithms for the analysis of large-scale time series data. We first develop a new fast algorithm to estimate the leverage scores of an autoregressive (AR) model in big data regimes. We show that the accuracy of approximations lies within $(1+\bigO{\varepsilon})$ of the true leverage scores with high probability. These theoretical results are subsequently exploited to develop an efficient algorithm, called LSAR, for fitting an appropriate AR model to big time series data. Our proposed algorithm is guaranteed, with high probability, to find the maximum likelihood estimates of the parameters of the underlying true AR model and has a worst case running time that significantly improves those of the state-of-the-art alternatives in big data regimes. Empirical results on large-scale synthetic as well as real data highly support the theoretical results and reveal the efficacy of this new approach.
△ Less
Submitted 30 October, 2021; v1 submitted 27 November, 2019;
originally announced November 2019.
-
The Importance of Environmental Factors in Forecasting Australian Power Demand
Authors:
Ali Eshragh,
Benjamin Ganim,
Terry Perkins,
Kasun Bandara
Abstract:
We develop a time series model to forecast weekly peak power demand for three main states of Australia for a yearly time-scale, and show the crucial role of environmental factors in improving the forecasts. More precisely, we construct a seasonal autoregressive integrated moving average (SARIMA) model and reinforce it by employing the exogenous environmental variables including, maximum temperatur…
▽ More
We develop a time series model to forecast weekly peak power demand for three main states of Australia for a yearly time-scale, and show the crucial role of environmental factors in improving the forecasts. More precisely, we construct a seasonal autoregressive integrated moving average (SARIMA) model and reinforce it by employing the exogenous environmental variables including, maximum temperature, minimum temperature, and solar exposure. The estimated hybrid SARIMA-regression model exhibits an excellent mean absolute percentage error (MAPE) of 3.41%. Moreover, our analysis demonstrates the importance of the environmental factors by showing a remarkable improvement of 46.3% in MAPE for the hybrid model over the crude SARIMA model which merely includes the power demand variables. In order to illustrate the efficacy of our model, we compare our outcome with the state-of-the-art machine learning methods in forecasting. The results reveal that our model outperforms the latter approach.
△ Less
Submitted 30 October, 2021; v1 submitted 2 November, 2019;
originally announced November 2019.
-
Demand Forecasting in the Presence of Systematic Events: Cases in Capturing Sales Promotions
Authors:
Mahdi Abolghasemi,
Ali Eshragh,
Jason Hurley,
Behnam Fahimnia
Abstract:
Reliable demand forecasts are critical for the effective supply chain management. Several endogenous and exogenous variables can influence the dynamics of demand, and hence a single statistical model that only consists of historical sales data is often insufficient to produce accurate forecasts. In practice, the forecasts generated by baseline statistical models are often judgmentally adjusted by…
▽ More
Reliable demand forecasts are critical for the effective supply chain management. Several endogenous and exogenous variables can influence the dynamics of demand, and hence a single statistical model that only consists of historical sales data is often insufficient to produce accurate forecasts. In practice, the forecasts generated by baseline statistical models are often judgmentally adjusted by forecasters to incorporate factors and information that are not incorporated in the baseline models. There are however systematic events whose effect can be effectively quantified and modeled to help minimize human intervention in adjusting the baseline forecasts. In this paper, we develop and test a novel regime-switching approach to quantify systematic information/events and objectively incorporate them into the baseline statistical model. Our simple yet practical and effective model can help limit forecast adjustments to only focus on the impact of less systematic events such as sudden climate change or dynamic market activities. The proposed model and approach is validated empirically using sales and promotional data from two Australian companies. Discussions focus on a thorough analysis of the forecasting and benchmarking results. Our analysis indicates that the proposed model can successfully improve the forecast accuracy when compared to the current industry practice which heavily relies on human judgment to factor in all types of information/events.
△ Less
Submitted 6 September, 2019;
originally announced September 2019.
-
Learning to Project in Multi-Objective Binary Linear Programming
Authors:
Alvaro Sierra-Altamiranda,
Hadi Charkhgard,
Iman Dayarian,
Ali Eshragh,
Sorna Javadi
Abstract:
In this paper, we investigate the possibility of improving the performance of multi-objective optimization solution approaches using machine learning techniques. Specifically, we focus on multi-objective binary linear programs and employ one of the most effective and recently developed criterion space search algorithms, the so-called KSA, during our study. This algorithm computes all nondominated…
▽ More
In this paper, we investigate the possibility of improving the performance of multi-objective optimization solution approaches using machine learning techniques. Specifically, we focus on multi-objective binary linear programs and employ one of the most effective and recently developed criterion space search algorithms, the so-called KSA, during our study. This algorithm computes all nondominated points of a problem with p objectives by searching on a projected criterion space, i.e., a (p-1)-dimensional criterion apace. We present an effective and fast learning approach to identify on which projected space the KSA should work. We also present several generic features/variables that can be used in machine learning techniques for identifying the best projected space. Finally, we present an effective bi-objective optimization based heuristic for selecting the best subset of the features to overcome the issue of overfitting in learning. Through an extensive computational study over 2000 instances of tri-objective Knapsack and Assignment problems, we demonstrate that an improvement of up to 12% in time can be achieved by the proposed learning method compared to a random selection of the projected space.
△ Less
Submitted 30 January, 2019;
originally announced January 2019.
-
Best subset selection in linear regression via bi-objective mixed integer linear programming
Authors:
Hadi Charkhgard,
Ali Eshragh
Abstract:
We study the problem of choosing the best subset of p features in linear regression given n observations. This problem naturally contains two objective functions including minimizing the amount of bias and minimizing the number of predictors. The existing approaches transform the problem into a single-objective optimization problem. We explain the main weaknesses of existing approaches, and to ove…
▽ More
We study the problem of choosing the best subset of p features in linear regression given n observations. This problem naturally contains two objective functions including minimizing the amount of bias and minimizing the number of predictors. The existing approaches transform the problem into a single-objective optimization problem. We explain the main weaknesses of existing approaches, and to overcome their drawbacks, we propose a bi-objective mixed integer linear programming approach. A computational study shows the efficacy of the proposed approach.
△ Less
Submitted 21 April, 2018;
originally announced April 2018.
