-
Concentration inequalities for high-dimensional linear processes with dependent innovations
Authors:
Eduardo Fonseca Mendes,
Fellipe Lopes
Abstract:
We develop concentration inequalities for the $l_\infty$ norm of vector linear processes with sub-Weibull, mixingale innovations. This inequality is used to obtain a concentration bound for the maximum entrywise norm of the lag-$h$ autocovariance matrix of linear processes. We apply these inequalities to sparse estimation of large-dimensional VAR(p) systems and heterocedasticity and autocorrelatio…
▽ More
We develop concentration inequalities for the $l_\infty$ norm of vector linear processes with sub-Weibull, mixingale innovations. This inequality is used to obtain a concentration bound for the maximum entrywise norm of the lag-$h$ autocovariance matrix of linear processes. We apply these inequalities to sparse estimation of large-dimensional VAR(p) systems and heterocedasticity and autocorrelation consistent (HAC) high-dimensional covariance estimation.
△ Less
Submitted 17 October, 2024; v1 submitted 23 July, 2023;
originally announced July 2023.
-
Regularized Estimation of High-Dimensional Vector AutoRegressions with Weakly Dependent Innovations
Authors:
Ricardo P. Masini,
Marcelo C. Medeiros,
Eduardo F. Mendes
Abstract:
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive models with heavy tailed, weakly dependent innovations with virtually no assumption o…
▽ More
There has been considerable advance in understanding the properties of sparse regularization procedures in high-dimensional models. In time series context, it is mostly restricted to Gaussian autoregressions or mixing sequences. We study oracle properties of LASSO estimation of weakly sparse vector-autoregressive models with heavy tailed, weakly dependent innovations with virtually no assumption on the conditional heteroskedasticity. In contrast to current literature, our innovation process satisfy an $L^1$ mixingale type condition on the centered conditional covariance matrices. This condition covers $L^1$-NED sequences and strong ($α$-) mixing sequences as particular examples.
△ Less
Submitted 11 June, 2021; v1 submitted 18 December, 2019;
originally announced December 2019.
-
Convergence Rates for Mixture-of-Experts
Authors:
Eduardo F. Mendes,
Wenxin Jiang
Abstract:
In mixtures-of-experts (ME) model, where a number of submodels (experts) are combined, there have been two longstanding problems: (i) how many experts should be chosen, given the size of the training data? (ii) given the total number of parameters, is it better to use a few very complex experts, or is it better to combine many simple experts? In this paper, we try to provide some insights to these…
▽ More
In mixtures-of-experts (ME) model, where a number of submodels (experts) are combined, there have been two longstanding problems: (i) how many experts should be chosen, given the size of the training data? (ii) given the total number of parameters, is it better to use a few very complex experts, or is it better to combine many simple experts? In this paper, we try to provide some insights to these problems through a theoretic study on a ME structure where $m$ experts are mixed, with each expert being related to a polynomial regression model of order $k$. We study the convergence rate of the maximum likelihood estimator (MLE), in terms of how fast the Kullback-Leibler divergence of the estimated density converges to the true density, when the sample size $n$ increases. The convergence rate is found to be dependent on both $m$ and $k$, and certain choices of $m$ and $k$ are found to produce optimal convergence rates. Therefore, these results shed light on the two aforementioned important problems: on how to choose $m$, and on how $m$ and $k$ should be compromised, for achieving good convergence rates.
△ Less
Submitted 1 November, 2011; v1 submitted 10 October, 2011;
originally announced October 2011.
