-
SGMM: Stochastic Approximation to Generalized Method of Moments
Authors:
Xiaohong Chen,
Sokbae Lee,
Yuan Liao,
Myung Hwan Seo,
Youngki Shin,
Myunghyun Song
Abstract:
We introduce a new class of algorithms, Stochastic Generalized Method of Moments (SGMM), for estimation and inference on (overidentified) moment restriction models. Our SGMM is a novel stochastic approximation alternative to the popular Hansen (1982) (offline) GMM, and offers fast and scalable implementation with the ability to handle streaming datasets in real time. We establish the almost sure c…
▽ More
We introduce a new class of algorithms, Stochastic Generalized Method of Moments (SGMM), for estimation and inference on (overidentified) moment restriction models. Our SGMM is a novel stochastic approximation alternative to the popular Hansen (1982) (offline) GMM, and offers fast and scalable implementation with the ability to handle streaming datasets in real time. We establish the almost sure convergence, and the (functional) central limit theorem for the inefficient online 2SLS and the efficient SGMM. Moreover, we propose online versions of the Durbin-Wu-Hausman and Sargan-Hansen tests that can be seamlessly integrated within the SGMM framework. Extensive Monte Carlo simulations show that as the sample size increases, the SGMM matches the standard (offline) GMM in terms of estimation accuracy and gains over computational efficiency, indicating its practical value for both large-scale and online datasets. We demonstrate the efficacy of our approach by a proof of concept using two well known empirical examples with large sample sizes.
△ Less
Submitted 30 October, 2023; v1 submitted 24 August, 2023;
originally announced August 2023.
-
Minimax Risk in Estimating Kink Threshold and Testing Continuity
Authors:
Javier Hidalgo,
Heejun Lee,
Jungyoon Lee,
Myung Hwan Seo
Abstract:
We derive a risk lower bound in estimating the threshold parameter without knowing whether the threshold regression model is continuous or not. The bound goes to zero as the sample size $ n $ grows only at the cube root rate. Motivated by this finding, we develop a continuity test for the threshold regression model and a bootstrap to compute its \textit{p}-values. The validity of the bootstrap is…
▽ More
We derive a risk lower bound in estimating the threshold parameter without knowing whether the threshold regression model is continuous or not. The bound goes to zero as the sample size $ n $ grows only at the cube root rate. Motivated by this finding, we develop a continuity test for the threshold regression model and a bootstrap to compute its \textit{p}-values. The validity of the bootstrap is established, and its finite sample property is explored through Monte Carlo simulations.
△ Less
Submitted 1 March, 2022;
originally announced March 2022.
-
Fast and Robust Online Inference with Stochastic Gradient Descent via Random Scaling
Authors:
Sokbae Lee,
Yuan Liao,
Myung Hwan Seo,
Youngki Shin
Abstract:
We develop a new method of online inference for a vector of parameters estimated by the Polyak-Ruppert averaging procedure of stochastic gradient descent (SGD) algorithms. We leverage insights from time series regression in econometrics and construct asymptotically pivotal statistics via random scaling. Our approach is fully operational with online data and is rigorously underpinned by a functiona…
▽ More
We develop a new method of online inference for a vector of parameters estimated by the Polyak-Ruppert averaging procedure of stochastic gradient descent (SGD) algorithms. We leverage insights from time series regression in econometrics and construct asymptotically pivotal statistics via random scaling. Our approach is fully operational with online data and is rigorously underpinned by a functional central limit theorem. Our proposed inference method has a couple of key advantages over the existing methods. First, the test statistic is computed in an online fashion with only SGD iterates and the critical values can be obtained without any resampling methods, thereby allowing for efficient implementation suitable for massive online data. Second, there is no need to estimate the asymptotic variance and our inference method is shown to be robust to changes in the tuning parameters for SGD algorithms in simulation experiments with synthetic data.
△ Less
Submitted 6 October, 2021; v1 submitted 6 June, 2021;
originally announced June 2021.
-
Inference for parameters identified by conditional moment restrictions using a generalized Bierens maximum statistic
Authors:
Xiaohong Chen,
Sokbae Lee,
Myung Hwan Seo,
Myunghyun Song
Abstract:
Many economic panel and dynamic models, such as rational behavior and Euler equations, imply that the parameters of interest are identified by conditional moment restrictions. We introduce a novel inference method without any prior information about which conditioning instruments are weak or irrelevant. Building on Bierens (1990), we propose penalized maximum statistics and combine bootstrap infer…
▽ More
Many economic panel and dynamic models, such as rational behavior and Euler equations, imply that the parameters of interest are identified by conditional moment restrictions. We introduce a novel inference method without any prior information about which conditioning instruments are weak or irrelevant. Building on Bierens (1990), we propose penalized maximum statistics and combine bootstrap inference with model selection. Our method optimizes asymptotic power by solving a data-dependent max-min problem for tuning parameter selection. Extensive Monte Carlo experiments, based on an empirical example, demonstrate the extent to which our inference procedure is superior to those available in the literature.
△ Less
Submitted 31 October, 2024; v1 submitted 25 August, 2020;
originally announced August 2020.
-
Robust inference for threshold regression models
Authors:
Javier Hidalgo,
Jungyoon Lee,
Myung Hwan Seo
Abstract:
This paper is concerned with inference in threshold regression models when the practitioners do not know whether at the threshold point the true specification has a kink or a jump. We nest previous works that assume either continuity or discontinuity at the threshold point and develop robust inference methods on the parameters of the model, which are valid under both specifications. In particular,…
▽ More
This paper is concerned with inference in threshold regression models when the practitioners do not know whether at the threshold point the true specification has a kink or a jump. We nest previous works that assume either continuity or discontinuity at the threshold point and develop robust inference methods on the parameters of the model, which are valid under both specifications. In particular, we found that the parameter values under the kink restriction are irregular points of the Hessian matrix of the expected Gaussian quasi-likelihood. This irregularity destroys the asymptotic normality and induces the non-standard cube root convergence rate for the threshold estimate. However, it also enables us to obtain the same asymptotic distribution as in Hansen (2000) for the quasi-likelihood ratio statistic for the unknown threshold up to an unknown scale parameter. We show that this scale parameter can be consistently estimated by a kernel method as long as no higher order kernel is used. Furthermore, we propose to construct confidence intervals for the unknown threshold by bootstrap test inversion, also known as grid bootstrap. Finite sample performances of the grid bootstrap confidence intervals are examined through Monte Carlo simulations. We also implement our procedure to an economic empirical application.
△ Less
Submitted 12 November, 2018; v1 submitted 2 February, 2017;
originally announced February 2017.
-
Local M-estimation with Discontinuous Criterion for Dependent and Limited Observations
Authors:
Myung Hwan Seo,
Taisuke Otsu
Abstract:
This paper examines asymptotic properties of local M-estimators under three sets of high-level conditions. These conditions are sufficiently general to cover the minimum volume predictive region, conditional maximum score estimator for a panel data discrete choice model, and many other widely used estimators in statistics and econometrics. Specifically, they allow for discontinuous criterion funct…
▽ More
This paper examines asymptotic properties of local M-estimators under three sets of high-level conditions. These conditions are sufficiently general to cover the minimum volume predictive region, conditional maximum score estimator for a panel data discrete choice model, and many other widely used estimators in statistics and econometrics. Specifically, they allow for discontinuous criterion functions of weakly dependent observations, which may be localized by kernel smoothing and contain nuisance parameters whose dimension may grow to infinity. Furthermore, the localization can occur around parameter values rather than around a fixed point and the observation may take limited values, which leads to set estimators. Our theory produces three different nonparametric cube root rates and enables valid inference for the local M-estimators, building on novel maximal inequalities for weakly dependent data. Our results include the standard cube root asymptotics as a special case. To illustrate the usefulness of our results, we verify our conditions for various examples such as the Hough transform estimator with diminishing bandwidth, maximum score-type set estimator, and many others.
△ Less
Submitted 9 October, 2016;
originally announced October 2016.
-
The Lasso for High-Dimensional Regression with a Possible Change-Point
Authors:
Sokbae Lee,
Myung Hwan Seo,
Youngki Shin
Abstract:
We consider a high-dimensional regression model with a possible change-point due to a covariate threshold and develop the Lasso estimator of regression coefficients as well as the threshold parameter. Our Lasso estimator not only selects covariates but also selects a model between linear and threshold regression models. Under a sparsity assumption, we derive non-asymptotic oracle inequalities for…
▽ More
We consider a high-dimensional regression model with a possible change-point due to a covariate threshold and develop the Lasso estimator of regression coefficients as well as the threshold parameter. Our Lasso estimator not only selects covariates but also selects a model between linear and threshold regression models. Under a sparsity assumption, we derive non-asymptotic oracle inequalities for both the prediction risk and the $\ell_1$ estimation loss for regression coefficients. Since the Lasso estimator selects variables simultaneously, we show that oracle inequalities can be established without pretesting the existence of the threshold effect. Furthermore, we establish conditions under which the estimation error of the unknown threshold parameter can be bounded by a nearly $n^{-1}$ factor even when the number of regressors can be much larger than the sample size ($n$). We illustrate the usefulness of our proposed estimation method via Monte Carlo simulations and an application to real data.
△ Less
Submitted 19 April, 2014; v1 submitted 21 September, 2012;
originally announced September 2012.
