Showing 1–50 of 55 results for author: Lv, J

Numerical method for the inverse scattering by random periodic structures

Authors: Yi Wang, Lei Lin, Junliang Lv

Abstract: Due to manufacturing defects or wear and tear, industrial components may have uncertainties. In order to evaluate the performance of machined components, it is crucial to quantify the uncertainty of the scattering surface. This brings up an important class of inverse scattering problems for random interface reconstruction. In this paper, we present an efficient numerical algorithm for the inverse… ▽ More Due to manufacturing defects or wear and tear, industrial components may have uncertainties. In order to evaluate the performance of machined components, it is crucial to quantify the uncertainty of the scattering surface. This brings up an important class of inverse scattering problems for random interface reconstruction. In this paper, we present an efficient numerical algorithm for the inverse scattering problem of acoustic-elastic interaction with random periodic interfaces. The proposed algorithm combines the Monte Carlo technique and the continuation method with respect to the wavenumber, which can accurately reconstruct the key statistics of random periodic interfaces from the measured data of the acoustic scattered field. In the implementation of our algorithm, a key two-step strategy is employed: Firstly, the elastic displacement field below the interface is determined by Tikhonov regularization based on the dynamic interface condition; Secondly, the profile function is iteratively updated and optimised using the Landweber method according to the kinematic interface condition. Such a algorithm does not require a priori information about the stochastic structures and performs well for both stationary Gaussian and non-Gaussian stochastic processes. Numerical experiments demonstrate the reliability and effectiveness of our proposed method. △ Less

Submitted 25 April, 2025; originally announced April 2025.

Comments: 26 pages, 15 figures

MSC Class: 74J25; 35R30; 65N21

An Adaptive Finite Element DtN Method for the Acoustic-Elastic Interaction Problem in Periodic Structures

Authors: Lei Lin, Junliang Lv

Abstract: Consider a time-harmonic acoustic plane wave incident onto an elastic body with an unbounded periodic surface. The medium above the surface is supposed to be filled with a homogeneous compressible inviscid air/fluid of constant mass density, while the elastic body is assumed to be isotropic and linear. By introducing the Dirichlet-to-Neumann (DtN) operators for acoustic and elastic waves simultane… ▽ More Consider a time-harmonic acoustic plane wave incident onto an elastic body with an unbounded periodic surface. The medium above the surface is supposed to be filled with a homogeneous compressible inviscid air/fluid of constant mass density, while the elastic body is assumed to be isotropic and linear. By introducing the Dirichlet-to-Neumann (DtN) operators for acoustic and elastic waves simultaneously, the model is formulated as an acoustic-elastic interaction problem in periodic structures. Based on a duality argument, an a posteriori error estimate is derived for the associated truncated finite element approximation. The a posteriori error estimate consists of the finite element approximation error and the truncation error of two different DtN operators, where the latter decays exponentially with respect to the truncation parameter. Based on the a posteriori error, an adaptive finite element algorithm is proposed for solving the acoustic-elastic interaction problem in periodic structures. Numerical experiments are presented to demonstrate the effectiveness of the proposed algorithm. △ Less

Submitted 24 April, 2025; originally announced April 2025.

Comments: 28 pages, 9 figures

MSC Class: 65N12; 65N15; 65N30

Alternately-optimized SNN method for acoustic scattering problem in unbounded domain

Authors: Haoming Song, Zhiqiang Sheng, Dong Wang, Junliang Lv

Abstract: In this paper, we propose a novel machine learning-based method to solve the acoustic scattering problem in unbounded domain. We first employ the Dirichlet-to-Neumann (DtN) operator to truncate the physically unbounded domain into a computable bounded domain. This transformation reduces the original scattering problem in the unbounded domain to a boundary value problem within the bounded domain. T… ▽ More In this paper, we propose a novel machine learning-based method to solve the acoustic scattering problem in unbounded domain. We first employ the Dirichlet-to-Neumann (DtN) operator to truncate the physically unbounded domain into a computable bounded domain. This transformation reduces the original scattering problem in the unbounded domain to a boundary value problem within the bounded domain. To solve this boundary value problem, we design a neural network with a subspace layer, where each neuron in this layer represents a basis function. Consequently, the approximate solution can be expressed by a linear combination of these basis functions. Furthermore, we introduce an innovative alternating optimization technique which alternately updates the basis functions and their linear combination coefficients respectively by training and least squares methods. In our method, we set the coefficients of basis functions to 1 and use a new loss function each time train the subspace. These innovations ensure that the subspace formed by these basis functions is truly optimized. We refer to this method as the alternately-optimized subspace method based on neural networks (AO-SNN). Extensive numerical experiments demonstrate that our new method can significantly reduce the relative $l^2$ error to $10^{-7}$ or lower, outperforming existing machine learning-based methods to the best of our knowledge. △ Less

Submitted 23 April, 2025; originally announced April 2025.

Comments: 30 pages, 8 figures

MSC Class: 65N22; 68T07 ACM Class: G.1.8; I.2.6

Asymptotic Theory of Eigenvectors for Latent Embeddings with Generalized Laplacian Matrices

Authors: Jianqing Fan, Yingying Fan, Jinchi Lv, Fan Yang, Diwen Yu

Abstract: Laplacian matrices are commonly employed in many real applications, encoding the underlying latent structural information such as graphs and manifolds. The use of the normalization terms naturally gives rise to random matrices with dependency. It is well-known that dependency is a major bottleneck of new random matrix theory (RMT) developments. To this end, in this paper, we formally introduce a c… ▽ More Laplacian matrices are commonly employed in many real applications, encoding the underlying latent structural information such as graphs and manifolds. The use of the normalization terms naturally gives rise to random matrices with dependency. It is well-known that dependency is a major bottleneck of new random matrix theory (RMT) developments. To this end, in this paper, we formally introduce a class of generalized (and regularized) Laplacian matrices, which contains the Laplacian matrix and the random adjacency matrix as a specific case, and suggest the new framework of the asymptotic theory of eigenvectors for latent embeddings with generalized Laplacian matrices (ATE-GL). Our new theory is empowered by the tool of generalized quadratic vector equation for dealing with RMT under dependency, and delicate high-order asymptotic expansions of the empirical spiked eigenvectors and eigenvalues based on local laws. The asymptotic normalities established for both spiked eigenvectors and eigenvalues will enable us to conduct precise inference and uncertainty quantification for applications involving the generalized Laplacian matrices with flexibility. We discuss some applications of the suggested ATE-GL framework and showcase its validity through some numerical examples. △ Less

Submitted 1 March, 2025; originally announced March 2025.

Comments: 104 pages, 12 figures

Asymptotic FDR Control with Model-X Knockoffs: Is Moments Matching Sufficient?

Authors: Yingying Fan, Lan Gao, Jinchi Lv, Xiaocong Xu

Abstract: We propose a unified theoretical framework for studying the robustness of the model-X knockoffs framework by investigating the asymptotic false discovery rate (FDR) control of the practically implemented approximate knockoffs procedure. This procedure deviates from the model-X knockoffs framework by substituting the true covariate distribution with a user-specified distribution that can be learned… ▽ More We propose a unified theoretical framework for studying the robustness of the model-X knockoffs framework by investigating the asymptotic false discovery rate (FDR) control of the practically implemented approximate knockoffs procedure. This procedure deviates from the model-X knockoffs framework by substituting the true covariate distribution with a user-specified distribution that can be learned using in-sample observations. By replacing the distributional exchangeability condition of the model-X knockoff variables with three conditions on the approximate knockoff statistics, we establish that the approximate knockoffs procedure achieves the asymptotic FDR control. Using our unified framework, we further prove that an arguably most popularly used knockoff variable generation method--the Gaussian knockoffs generator based on the first two moments matching--achieves the asymptotic FDR control when the two-moment-based knockoff statistics are employed in the knockoffs inference procedure. For the first time in the literature, our theoretical results justify formally the effectiveness and robustness of the Gaussian knockoffs generator. Simulation and real data examples are conducted to validate the theoretical findings. △ Less

Submitted 9 February, 2025; originally announced February 2025.

Comments: 90 pages

Precise Asymptotics and Refined Regret of Variance-Aware UCB

Authors: Yingying Fan, Yuxuan Han, Jinchi Lv, Xiaocong Xu, Zhengyuan Zhou

Abstract: In this paper, we study the behavior of the Upper Confidence Bound-Variance (UCB-V) algorithm for the Multi-Armed Bandit (MAB) problems, a variant of the canonical Upper Confidence Bound (UCB) algorithm that incorporates variance estimates into its decision-making process. More precisely, we provide an asymptotic characterization of the arm-pulling rates for UCB-V, extending recent results for the… ▽ More In this paper, we study the behavior of the Upper Confidence Bound-Variance (UCB-V) algorithm for the Multi-Armed Bandit (MAB) problems, a variant of the canonical Upper Confidence Bound (UCB) algorithm that incorporates variance estimates into its decision-making process. More precisely, we provide an asymptotic characterization of the arm-pulling rates for UCB-V, extending recent results for the canonical UCB in Kalvit and Zeevi (2021) and Khamaru and Zhang (2024). In an interesting contrast to the canonical UCB, our analysis reveals that the behavior of UCB-V can exhibit instability, meaning that the arm-pulling rates may not always be asymptotically deterministic. Besides the asymptotic characterization, we also provide non-asymptotic bounds for the arm-pulling rates in the high probability regime, offering insights into the regret analysis. As an application of this high probability result, we establish that UCB-V can achieve a more refined regret bound, previously unknown even for more complicate and advanced variance-aware online decision-making algorithms. △ Less

Submitted 16 February, 2025; v1 submitted 11 December, 2024; originally announced December 2024.

Exogenous Randomness Empowering Random Forests

Authors: Tianxing Mei, Yingying Fan, Jinchi Lv

Abstract: We offer theoretical and empirical insights into the impact of exogenous randomness on the effectiveness of random forests with tree-building rules independent of training data. We formally introduce the concept of exogenous randomness and identify two types of commonly existing randomness: Type I from feature subsampling, and Type II from tie-breaking in tree-building processes. We develop non-as… ▽ More We offer theoretical and empirical insights into the impact of exogenous randomness on the effectiveness of random forests with tree-building rules independent of training data. We formally introduce the concept of exogenous randomness and identify two types of commonly existing randomness: Type I from feature subsampling, and Type II from tie-breaking in tree-building processes. We develop non-asymptotic expansions for the mean squared error (MSE) for both individual trees and forests and establish sufficient and necessary conditions for their consistency. In the special example of the linear regression model with independent features, our MSE expansions are more explicit, providing more understanding of the random forests' mechanisms. It also allows us to derive an upper bound on the MSE with explicit consistency rates for trees and forests. Guided by our theoretical findings, we conduct simulations to further explore how exogenous randomness enhances random forest performance. Our findings unveil that feature subsampling reduces both the bias and variance of random forests compared to individual trees, serving as an adaptive mechanism to balance bias and variance. Furthermore, our results reveal an intriguing phenomenon: the presence of noise features can act as a "blessing" in enhancing the performance of random forests thanks to feature subsampling. △ Less

Submitted 12 November, 2024; originally announced November 2024.

Comments: 103 pages, 10 figures

Lagrangian Mean Curvature Flow in Pseudo-Euclidean Space II

Authors: Shanshan Li, Jiaru Lv, Rongli Huang

Abstract: In this paper, we consider the mean curvature flow of entire Lagrangian graphs with initial data in the pseudo-Euclidean space, which is related to the special Lagrangian parabolic equation. We show that the parabolic equation \eqref{11} has a smooth solution $u(x,t)$ for three corresponding nonlinear equations between the Monge-Amp$\grave{e}$re type equation($τ=0$) and the special Lagrangian para… ▽ More In this paper, we consider the mean curvature flow of entire Lagrangian graphs with initial data in the pseudo-Euclidean space, which is related to the special Lagrangian parabolic equation. We show that the parabolic equation \eqref{11} has a smooth solution $u(x,t)$ for three corresponding nonlinear equations between the Monge-Amp$\grave{e}$re type equation($τ=0$) and the special Lagrangian parabolic equation($τ=\fracπ{2}$). Furthermore, we get the bound of $D^lu$, $l=\{3,4,5,\cdots\}$ for $τ=\fracπ{4}$ and the decay estimates of the higher order derivatives when $0<τ<\fracπ{4}$ and $\fracπ{4}<τ<\fracπ{2}$. We also prove that $u(x,t)$ converges to smooth self-expanding solutions of \eqref{12}. △ Less

Submitted 23 October, 2024; originally announced October 2024.

Mirror descent method for stochastic multi-objective optimization

Authors: Linxi Yang, Liping Tang, Jiahao Lv, Yuehong He, Xinmin Yang

Abstract: Stochastic multi-objective optimization (SMOO) has recently emerged as a powerful framework for addressing machine learning problems with multiple objectives. The bias introduced by the nonlinearity of the subproblem solution mapping complicates the convergence analysis of multi-gradient methods. In this paper, we propose a novel SMOO method called the Multi-gradient Stochastic Mirror Descent (MSM… ▽ More Stochastic multi-objective optimization (SMOO) has recently emerged as a powerful framework for addressing machine learning problems with multiple objectives. The bias introduced by the nonlinearity of the subproblem solution mapping complicates the convergence analysis of multi-gradient methods. In this paper, we propose a novel SMOO method called the Multi-gradient Stochastic Mirror Descent (MSMD) method, which incorporates stochastic mirror descent method to solve the SMOO subproblem, providing convergence guarantees. By selecting an appropriate Bregman function, our method enables analytical solutions of the weighting vector and requires only a single gradient sample at each iteration. We demonstrate the sublinear convergence rate of our MSMD method under four different inner and outer step setups. For SMOO with preferences, we propose a variant of MSMD method and demonstrate its convergence rate. Through extensive numerical experiments, we compare our method with both stochastic descent methods based on weighted sum and state-of-the-art SMOO methods. Our method consistently outperforms these methods in terms of generating superior Pareto fronts on benchmark test functions while also achieving competitive results in neural network training. △ Less

Submitted 9 October, 2024; originally announced October 2024.

Factorization method for inverse elastic cavity scattering

Authors: Shuxin Li, Junliang Lv, Yi Wang

Abstract: This paper is concerned with the inverse elastic scattering problem to determine the shape and location of an elastic cavity. By establishing a one-to-one correspondence between the Herglotz wave function and its kernel, we introduce the far-field operator which is crucial in the factorization method. We present a theoretical factorization of the far-field operator and rigorously prove the propert… ▽ More This paper is concerned with the inverse elastic scattering problem to determine the shape and location of an elastic cavity. By establishing a one-to-one correspondence between the Herglotz wave function and its kernel, we introduce the far-field operator which is crucial in the factorization method. We present a theoretical factorization of the far-field operator and rigorously prove the properties of its associated operators involved in the factorization. Unlike the Dirichlet problem where the boundary integral operator of the single-layer potential involved in the factorization of the far-field operator is weakly singular, the boundary integral operator of the conormal derivative of the double-layer potential involved in the factorization of the far-field operator with Neumann boundary conditions is hypersingular, which forces us to prove that this operator is isomorphic using Fredholm's theorem. Meanwhile, we present theoretical analyses of the factorization method for various illumination and measurement cases, including compression-wave illumination and compression-wave measurement, shear-wave illumination and shear-wave measurement, and full-wave illumination and full-wave measurement. In addition, we also consider the limited aperture problem and provide a rigorous theoretical analysis of the factorization method in this case. Numerous numerical experiments are carried out to demonstrate the effectiveness of the proposed method, and to analyze the influence of various factors, such as polarization direction, frequency, wavenumber, and multi-scale scatterers on the reconstructed results. △ Less

Submitted 14 September, 2024; originally announced September 2024.

SOFARI: High-Dimensional Manifold-Based Inference

Authors: Zemin Zheng, Xin Zhou, Yingying Fan, Jinchi Lv

Abstract: Multi-task learning is a widely used technique for harnessing information from various tasks. Recently, the sparse orthogonal factor regression (SOFAR) framework, based on the sparse singular value decomposition (SVD) within the coefficient matrix, was introduced for interpretable multi-task learning, enabling the discovery of meaningful latent feature-response association networks across differen… ▽ More Multi-task learning is a widely used technique for harnessing information from various tasks. Recently, the sparse orthogonal factor regression (SOFAR) framework, based on the sparse singular value decomposition (SVD) within the coefficient matrix, was introduced for interpretable multi-task learning, enabling the discovery of meaningful latent feature-response association networks across different layers. However, conducting precise inference on the latent factor matrices has remained challenging due to the orthogonality constraints inherited from the sparse SVD constraints. In this paper, we suggest a novel approach called the high-dimensional manifold-based SOFAR inference (SOFARI), drawing on the Neyman near-orthogonality inference while incorporating the Stiefel manifold structure imposed by the SVD constraints. By leveraging the underlying Stiefel manifold structure that is crucial to enabling inference, SOFARI provides easy-to-use bias-corrected estimators for both latent left factor vectors and singular values, for which we show to enjoy the asymptotic mean-zero normal distributions with estimable variances. We introduce two SOFARI variants to handle strongly and weakly orthogonal latent factors, where the latter covers a broader range of applications. We illustrate the effectiveness of SOFARI and justify our theoretical results through simulation examples and a real data application in economic forecasting. △ Less

Submitted 1 July, 2025; v1 submitted 26 September, 2023; originally announced September 2023.

Comments: 208 pages, 2 figures, to appear in Journal of the American Statistical Association

ARK: Robust Knockoffs Inference with Coupling

Authors: Yingying Fan, Lan Gao, Jinchi Lv

Abstract: We investigate the robustness of the model-X knockoffs framework with respect to the misspecified or estimated feature distribution. We achieve such a goal by theoretically studying the feature selection performance of a practically implemented knockoffs algorithm, which we name as the approximate knockoffs (ARK) procedure, under the measures of the false discovery rate (FDR) and $k$-familywise er… ▽ More We investigate the robustness of the model-X knockoffs framework with respect to the misspecified or estimated feature distribution. We achieve such a goal by theoretically studying the feature selection performance of a practically implemented knockoffs algorithm, which we name as the approximate knockoffs (ARK) procedure, under the measures of the false discovery rate (FDR) and $k$-familywise error rate ($k$-FWER). The approximate knockoffs procedure differs from the model-X knockoffs procedure only in that the former uses the misspecified or estimated feature distribution. A key technique in our theoretical analyses is to couple the approximate knockoffs procedure with the model-X knockoffs procedure so that random variables in these two procedures can be close in realizations. We prove that if such coupled model-X knockoffs procedure exists, the approximate knockoffs procedure can achieve the asymptotic FDR or $k$-FWER control at the target level. We showcase three specific constructions of such coupled model-X knockoff variables, verifying their existence and justifying the robustness of the model-X knockoffs framework. Additionally, we formally connect our concept of knockoff variable coupling to a type of Wasserstein distance. △ Less

Submitted 4 June, 2024; v1 submitted 10 July, 2023; originally announced July 2023.

Comments: 105 pages

Model Predictive Control with Reach-avoid Analysis

Authors: Dejin Ren, Wanli Lu, Jidong Lv, Lijun Zhang, Bai Xue

Abstract: In this paper we investigate the optimal controller synthesis problem, so that the system under the controller can reach a specified target set while satisfying given constraints. Existing model predictive control (MPC) methods learn from a set of discrete states visited by previous (sub-)optimized trajectories and thus result in computationally expensive mixed-integer nonlinear optimization. In t… ▽ More In this paper we investigate the optimal controller synthesis problem, so that the system under the controller can reach a specified target set while satisfying given constraints. Existing model predictive control (MPC) methods learn from a set of discrete states visited by previous (sub-)optimized trajectories and thus result in computationally expensive mixed-integer nonlinear optimization. In this paper a novel MPC method is proposed based on reach-avoid analysis to solve the controller synthesis problem iteratively. The reach-avoid analysis is concerned with computing a reach-avoid set which is a set of initial states such that the system can reach the target set successfully. It not only provides terminal constraints, which ensure feasibility of MPC, but also expands discrete states in existing methods into a continuous set (i.e., reach-avoid sets) and thus leads to nonlinear optimization which is more computationally tractable online due to the absence of integer variables. Finally, we evaluate the proposed method and make comparisons with state-of-the-art ones based on several examples. △ Less

Submitted 21 June, 2023; v1 submitted 15 May, 2023; originally announced May 2023.

Existence of nontrivial solutions for critical biharmonic equations with logarithmic term

Authors: Qihan He, Juntao Lv, Zongyan Lv, Tong Wu

Abstract: In this paper, we consider the existence of nontrivial solutions to the following critical biharmonic problem with a logarithmic term \begin{equation*} \begin{cases} Δ^2 u=μΔu+λu+|u|^{2^{**}-2}u+τu\log u^2, \ \ x\inΩ, u|_{\partial Ω}=\frac{\partial u}{\partial n}|_{\partialΩ}=0, \end{cases} \end{equation*} where $μ,λ,τ\in \mathbb{R}$, $|μ|+|τ|\ne 0$, $Δ^2=ΔΔ$ denotes the iterated N-dimensional Lap… ▽ More In this paper, we consider the existence of nontrivial solutions to the following critical biharmonic problem with a logarithmic term \begin{equation*} \begin{cases} Δ^2 u=μΔu+λu+|u|^{2^{**}-2}u+τu\log u^2, \ \ x\inΩ, u|_{\partial Ω}=\frac{\partial u}{\partial n}|_{\partialΩ}=0, \end{cases} \end{equation*} where $μ,λ,τ\in \mathbb{R}$, $|μ|+|τ|\ne 0$, $Δ^2=ΔΔ$ denotes the iterated N-dimensional Laplacian, $Ω\subset \mathbb{R}^{N}$ is a bounded domain with smooth boundary $\partial Ω$, $2^{**}=\frac{2N}{N-4}(N\ge5)$ is the critical Sobolev exponent for the embedding $H_{0}^{2}(Ω)\hookrightarrow L^{2^{**}}(Ω)$ and $H_0^2 (Ω)$ is the closure of $C_0^ \infty (Ω)$ under the norm $|| u ||:=(\int_Ω|Δu|^2)^\frac{1}{2}$. The uncertainty of the sign of $s\log s^2$ in $(0,+\infty)$ has some interest in itself. To know which of the three terms $μΔu$, $λu$ and $τu \log u^2$ has a greater influence on the existence of nontrivial weak solutions, we prove the existence of nontrivial weak solutions to the above problem for $N\ge5$ under some assumptions of $λ, μ$ and $τ$. △ Less

Submitted 14 March, 2023; originally announced March 2023.

MSC Class: 2020: 35A01; 35A15; 35B33; 35D30; 35G30

SIMPLE-RC: Group Network Inference with Non-Sharp Nulls and Weak Signals

Authors: Jianqing Fan, Yingying Fan, Jinchi Lv, Fan Yang

Abstract: Large-scale network inference with uncertainty quantification has important applications in natural, social, and medical sciences. The recent work of Fan, Fan, Han and Lv (2022) introduced a general framework of statistical inference on membership profiles in large networks (SIMPLE) for testing the sharp null hypothesis that a pair of given nodes share the same membership profiles. In real applica… ▽ More Large-scale network inference with uncertainty quantification has important applications in natural, social, and medical sciences. The recent work of Fan, Fan, Han and Lv (2022) introduced a general framework of statistical inference on membership profiles in large networks (SIMPLE) for testing the sharp null hypothesis that a pair of given nodes share the same membership profiles. In real applications, there are often groups of nodes under investigation that may share similar membership profiles at the presence of relatively weaker signals than the setting considered in SIMPLE. To address these practical challenges, in this paper we propose a SIMPLE method with random coupling (SIMPLE-RC) for testing the non-sharp null hypothesis that a group of given nodes share similar (not necessarily identical) membership profiles under weaker signals. Utilizing the idea of random coupling, we construct our test as the maximum of the SIMPLE tests for subsampled node pairs from the group. Such technique reduces significantly the correlation among individual SIMPLE tests while largely maintaining the power, enabling delicate analysis on the asymptotic distributions of the SIMPLE-RC test. Our method and theory cover both the cases with and without node degree heterogeneity. These new theoretical developments are empowered by a second-order expansion of spiked eigenvectors under the $\ell_\infty$-norm, built upon our work for random matrices with weak spikes. Our theoretical results and the practical advantages of the newly suggested method are demonstrated through several simulation and real data examples. △ Less

Submitted 31 October, 2022; originally announced November 2022.

Comments: 71 pages, 4 figures

FACT: High-Dimensional Random Forests Inference

Authors: Chien-Ming Chi, Yingying Fan, Jinchi Lv

Abstract: Quantifying the usefulness of individual features in random forests learning can greatly enhance its interpretability. Existing studies have shown that some popularly used feature importance measures for random forests suffer from the bias issue. In addition, there lack comprehensive size and power analyses for most of these existing methods. In this paper, we approach the problem via hypothesis t… ▽ More Quantifying the usefulness of individual features in random forests learning can greatly enhance its interpretability. Existing studies have shown that some popularly used feature importance measures for random forests suffer from the bias issue. In addition, there lack comprehensive size and power analyses for most of these existing methods. In this paper, we approach the problem via hypothesis testing, and suggest a framework of the self-normalized feature-residual correlation test (FACT) for evaluating the significance of a given feature in the random forests model with bias-resistance property, where our null hypothesis concerns whether the feature is conditionally independent of the response given all other features. Such an endeavor on random forests inference is empowered by some recent developments on high-dimensional random forests consistency. Under a fairly general high-dimensional nonparametric model setting with dependent features, we formally establish that FACT can provide theoretically justified feature importance test with controlled type I error and enjoy appealing power property. The theoretical results and finite-sample advantages of the newly suggested method are illustrated with several simulation examples and an economic forecasting application. △ Less

Submitted 12 November, 2023; v1 submitted 4 July, 2022; originally announced July 2022.

Comments: 42 pages, 3 figures

Jackknife Partially Linear Model Averaging for the Conditional Quantile Prediction

Authors: Jing Lv

Abstract: Estimating the conditional quantile of the interested variable with respect to changes in the covariates is frequent in many economical applications as it can offer a comprehensive insight. In this paper, we propose a novel semiparametric model averaging to predict the conditional quantile even if all models under consideration are potentially misspecified. Specifically, we first build a series of… ▽ More Estimating the conditional quantile of the interested variable with respect to changes in the covariates is frequent in many economical applications as it can offer a comprehensive insight. In this paper, we propose a novel semiparametric model averaging to predict the conditional quantile even if all models under consideration are potentially misspecified. Specifically, we first build a series of non-nested partially linear sub-models, each with different nonlinear component. Then a leave-one-out cross-validation criterion is applied to choose the model weights. Under some regularity conditions, we have proved that the resulting model averaging estimator is asymptotically optimal in terms of minimizing the out-of-sample average quantile prediction error. Our modelling strategy not only effectively avoids the problem of specifying which a covariate should be nonlinear when one fits a partially linear model, but also results in a more accurate prediction than traditional model-based procedures because of the optimality of the selected weights by the cross-validation criterion. Simulation experiments and an illustrative application show that our proposed model averaging method is superior to other commonly used alternatives. △ Less

Submitted 7 June, 2022; v1 submitted 19 March, 2022; originally announced March 2022.

An Improved Reinforcement Learning Algorithm for Learning to Branch

Authors: Qingyu Qu, Xijun Li, Yunfan Zhou, Jia Zeng, Mingxuan Yuan, Jie Wang, Jinhu Lv, Kexin Liu, Kun Mao

Abstract: Most combinatorial optimization problems can be formulated as mixed integer linear programming (MILP), in which branch-and-bound (B\&B) is a general and widely used method. Recently, learning to branch has become a hot research topic in the intersection of machine learning and combinatorial optimization. In this paper, we propose a novel reinforcement learning-based B\&B algorithm. Similar to offl… ▽ More Most combinatorial optimization problems can be formulated as mixed integer linear programming (MILP), in which branch-and-bound (B\&B) is a general and widely used method. Recently, learning to branch has become a hot research topic in the intersection of machine learning and combinatorial optimization. In this paper, we propose a novel reinforcement learning-based B\&B algorithm. Similar to offline reinforcement learning, we initially train on the demonstration data to accelerate learning massively. With the improvement of the training effect, the agent starts to interact with the environment with its learned policy gradually. It is critical to improve the performance of the algorithm by determining the mixing ratio between demonstration and self-generated data. Thus, we propose a prioritized storage mechanism to control this ratio automatically. In order to improve the robustness of the training process, a superior network is additionally introduced based on Double DQN, which always serves as a Q-network with competitive performance. We evaluate the performance of the proposed algorithm over three public research benchmarks and compare it against strong baselines, including three classical heuristics and one state-of-the-art imitation learning-based branching algorithm. The results show that the proposed algorithm achieves the best performance among compared algorithms and possesses the potential to improve B\&B algorithm performance continuously. △ Less

Submitted 16 January, 2022; originally announced January 2022.

High-Dimensional Knockoffs Inference for Time Series Data

Authors: Chien-Ming Chi, Yingying Fan, Ching-Kang Ing, Jinchi Lv

Abstract: We make some initial attempt to establish the theoretical and methodological foundation for the model-X knockoffs inference for time series data. We suggest the method of time series knockoffs inference (TSKI) by exploiting the ideas of subsampling and e-values to address the difficulty caused by the serial dependence. We also generalize the robust knockoffs inference in Barber, Candès, and Samwor… ▽ More We make some initial attempt to establish the theoretical and methodological foundation for the model-X knockoffs inference for time series data. We suggest the method of time series knockoffs inference (TSKI) by exploiting the ideas of subsampling and e-values to address the difficulty caused by the serial dependence. We also generalize the robust knockoffs inference in Barber, Candès, and Samworth to the time series setting to relax the assumption of known covariate distribution required by model-X knockoffs, since such an assumption is overly stringent for time series data. We establish sufficient conditions under which TSKI achieves the asymptotic false discovery rate (FDR) control. Our technical analysis reveals the effects of serial dependence and unknown covariate distribution on the FDR control. We conduct a power analysis of TSKI using the Lasso coefficient difference knockoff statistic under the generalized linear time series models. The finite-sample performance of TSKI is illustrated with several simulation examples and an economic inflation study. △ Less

Submitted 28 February, 2025; v1 submitted 18 December, 2021; originally announced December 2021.

Comments: 106 pages, 6 figures

MSC Class: 62P20; Stats.ML ACM Class: A.0

Journal ref: Journal of the American Statistical Association 2025

Dimension-Free Average Treatment Effect Inference with Deep Neural Networks

Authors: Xinze Du, Yingying Fan, Jinchi Lv, Tianshu Sun, Patrick Vossler

Abstract: This paper investigates the estimation and inference of the average treatment effect (ATE) using deep neural networks (DNNs) in the potential outcomes framework. Under some regularity conditions, the observed response can be formulated as the response of a mean regression problem with both the confounding variables and the treatment indicator as the independent variables. Using such formulation, w… ▽ More This paper investigates the estimation and inference of the average treatment effect (ATE) using deep neural networks (DNNs) in the potential outcomes framework. Under some regularity conditions, the observed response can be formulated as the response of a mean regression problem with both the confounding variables and the treatment indicator as the independent variables. Using such formulation, we investigate two methods for ATE estimation and inference based on the estimated mean regression function via DNN regression using a specific network architecture. We show that both DNN estimates of ATE are consistent with dimension-free consistency rates under some assumptions on the underlying true mean regression model. Our model assumptions accommodate the potentially complicated dependence structure of the observed response on the covariates, including latent factors and nonlinear interactions between the treatment indicator and confounding variables. We also establish the asymptotic normality of our estimators based on the idea of sample splitting, ensuring precise inference and uncertainty quantification. Simulation studies and real data application justify our theoretical findings and support our DNN estimation and inference methods. △ Less

Submitted 2 December, 2021; originally announced December 2021.

Comments: 56 pages, 22 figures

Crashworthiness design of 3D lattice-structure filled thin-walled tubes based on data mining

Authors: Jiyuan Lv, Zhonghao Bai, Xianping Du, Feng Zhu, Clifford C. Chou, Binhui Jiang, Shiwei Xu

Abstract: Lattice structures and thin-walled tubes are two types of energy-absorbers widely studied and applied in engineering practice. In this study, a new type of lattice-structure filled thin-walled tube (LFT) was proposed. In this new type of LFT, a BCC-Z lattice structure was filled into a square thin-walled tube. Then using data mining, a 3-D geometric design with five design variables was conducted… ▽ More Lattice structures and thin-walled tubes are two types of energy-absorbers widely studied and applied in engineering practice. In this study, a new type of lattice-structure filled thin-walled tube (LFT) was proposed. In this new type of LFT, a BCC-Z lattice structure was filled into a square thin-walled tube. Then using data mining, a 3-D geometric design with five design variables was conducted on this new LFT. Using Latin Hypercubic sampling algorithm, 150 design cases were generated. Numerical models were then developed to simulate their crush behavior, and the simulation dataset was used for data mining. The results showed that (1) Filling the BBC-Z lattice structure into a thin-walled tube can significantly improve the energy absorption (EA) capacity of the structure. (2) The decision trees generated in the data mining process indicated that the rod diameter d of lattice structure is the key design variable that has most significant impact on EA, followed by m and n. (3) The design rules to build LFTs with high EA efficiency (SEA>=16 kJ/kg and CFE>=45%), high total EA (SEA>=16 kJ/kg and EA>=6 kJ), and lightweight (SEA>=16 kJ/kg and Mass<=0.45 kg) were obtained from decision trees. The ideal configurations of LFT corresponding to these three objectives are: d>2 mm, n>2 and m>3 for high EA efficiency; d>2 mm, n>2 and m>3 for high total EA; and d>2 mm, n>2, m<=4 and t<=1.7 mm for lightweight. △ Less

Submitted 1 October, 2021; originally announced October 2021.

Distributed Nash Equilibrium Seeking in Consistency-Constrained Multi-Coalition Games

Authors: Jialing Zhou, Yuezu Lv, Guanghui Wen, Jinhu Lv, Dezhi Zheng

Abstract: Distributed Nash equilibrium (NE) seeking problem for multi-coalition games has attracted increasing attention in recent years, but the research mainly focuses on the case without agreement demand within coalitions. This paper considers a class of networked games among multiple coalitions where each coalition contains multiple agents that cooperate to minimize the sum of their costs, subject to th… ▽ More Distributed Nash equilibrium (NE) seeking problem for multi-coalition games has attracted increasing attention in recent years, but the research mainly focuses on the case without agreement demand within coalitions. This paper considers a class of networked games among multiple coalitions where each coalition contains multiple agents that cooperate to minimize the sum of their costs, subject to the demand of reaching an agreement on their state values. Furthermore, the underlying network topology among the agents does not need to be balanced. To achieve the goal of NE seeking within such a context, two estimates are constructed for each agent, namely, an estimate of partial derivatives of the cost function and an estimate of global state values, based on which, an iterative state updating law is elaborately designed. Linear convergence of the proposed algorithm is demonstrated. It is shown that the consistency-constrained multi-coalition games investigated in this paper put the well-studied networked games among individual players and distributed optimization in a unified framework, and the proposed algorithm can easily degenerate into solutions to these problems. △ Less

Submitted 9 December, 2021; v1 submitted 19 June, 2021; originally announced June 2021.

Explicit continuation methods with L-BFGS updating formulas for linearly constrained optimization problems

Authors: Xin-long Luo, Jia-hui Lv, Hang Xiao

Abstract: This paper considers an explicit continuation method with the trusty time-stepping scheme and the limited-memory BFGS (L-BFGS) updating formula (Eptctr) for the linearly constrained optimization problem. At every iteration, Eptctr only involves three pairs of the inner product of vector and one matrix-vector product, other than the traditional and representative optimization method such as the seq… ▽ More This paper considers an explicit continuation method with the trusty time-stepping scheme and the limited-memory BFGS (L-BFGS) updating formula (Eptctr) for the linearly constrained optimization problem. At every iteration, Eptctr only involves three pairs of the inner product of vector and one matrix-vector product, other than the traditional and representative optimization method such as the sequential quadratic programming (SQP) or the latest continuation method such as Ptctr \cite{LLS2020}, which needs to solve a quadratic programming subproblem (SQP) or a linear system of equations (Ptctr). Thus, Eptctr can save much more computational time than SQP or Ptctr. Numerical results also show that the consumed time of EPtctr is about one tenth of that of Ptctr or one fifteenth to 0.4 percent of that of SQP. Furthermore, Eptctr can save the storage space of an $(n+m) \times (n+m)$ large-scale matrix, in comparison to SQP. The required memory of Eptctr is about one fifth of that of SQP. Finally, we also give the global convergence analysis of the new method under the standard assumptions. △ Less

Submitted 18 January, 2021; originally announced January 2021.

Comments: arXiv admin note: substantial text overlap with arXiv:2012.14808; text overlap with arXiv:2005.05965

Explicit pseudo-transient continuation and the trust-region updating strategy for unconstrained optimization

Authors: Xin-long Luo, Hang Xiao, Jia-hui Lv, Sen Zhang

Abstract: This paper considers an explicit continuation method and the trust-region updating strategy for the unconstrained optimization problem. Moreover, in order to improve its computational efficiency and robustness, the new method uses the switching preconditioning technique. In the well-conditioned phase, the new method uses the L-BFGS method as the preconditioning technique in order to improve its co… ▽ More This paper considers an explicit continuation method and the trust-region updating strategy for the unconstrained optimization problem. Moreover, in order to improve its computational efficiency and robustness, the new method uses the switching preconditioning technique. In the well-conditioned phase, the new method uses the L-BFGS method as the preconditioning technique in order to improve its computational efficiency. Otherwise, the new method uses the inverse of the Hessian matrix as the pre-conditioner in order to improve its robustness. Numerical results aslo show that the new method is more robust and faster than the traditional optimization method such as the trust-region method and the line search method. The computational time of the new method is about one percent of that of the trust-region method (the subroutine fminunc.m of the MATLAB2019a environment, it is set by the trust-region method) or one fifth of that the line search method (fminunc.m is set by the quasi-Newton method) for the large-scale problem. Finally, the global convergence analysis of the new method is also given. △ Less

Submitted 13 February, 2021; v1 submitted 29 December, 2020; originally announced December 2020.

Continuation Newton methods with the residual trust-region time-stepping scheme for nonlinear equations

Authors: Xin-long Luo, Hang Xiao, Jia-hui Lv

Abstract: For nonlinear equations, the homotopy methods (continuation methods) are popular in engineering fields since their convergence regions are large and they are quite reliable to find a solution. The disadvantage of the classical homotopy methods is that their computational time is heavy since they need to solve many auxiliary nonlinear systems during the intermediate continuation processes. In order… ▽ More For nonlinear equations, the homotopy methods (continuation methods) are popular in engineering fields since their convergence regions are large and they are quite reliable to find a solution. The disadvantage of the classical homotopy methods is that their computational time is heavy since they need to solve many auxiliary nonlinear systems during the intermediate continuation processes. In order to overcome this shortcoming, we consider the special explicit continuation Newton method with the residual trust-region time-stepping scheme for this problem. According to our numerical experiments, the new method is more robust and faster to find the required solution of the real-world problem than the traditional optimization method (the built-in subroutine fsolve.m of the MATLAB environment) and the homotopy continuation methods(HOMPACK90 and NAClab). Furthermore, we analyze the global convergence and the local superlinear convergence of the new method. △ Less

Submitted 26 March, 2021; v1 submitted 3 June, 2020; originally announced June 2020.

Continuation Method with the Trusty Time-stepping Scheme for Linearly Constrained Optimization with Noisy Data

Authors: Xin-long Luo, Jia-hui Lv, Geng Sun

Abstract: The nonlinear optimization problem with linear constraints has many applications in engineering fields such as the visual-inertial navigation and localization of an unmanned aerial vehicle maintaining the horizontal flight. In order to solve this practical problem efficiently, this paper constructs a continuation method with the trusty time-stepping scheme for the linearly equality-constrained opt… ▽ More The nonlinear optimization problem with linear constraints has many applications in engineering fields such as the visual-inertial navigation and localization of an unmanned aerial vehicle maintaining the horizontal flight. In order to solve this practical problem efficiently, this paper constructs a continuation method with the trusty time-stepping scheme for the linearly equality-constrained optimization problem at every sampling time. At every iteration, the new method only solves a system of linear equations other than the traditional optimization method such as the sequential quadratic programming (SQP) method, which needs to solve a quadratic programming subproblem. Consequently, the new method can save much more computational time than SQP. Numerical results show that the new method works well for this problem and its consumed time is about one fifth of that of SQP (the built-in subroutine fmincon.m of the MATLAB2018a environment) or that of the traditional dynamical method (the built-in subroutine ode15s.m of the MATLAB2018a environment). Furthermore, we also give the global convergence analysis of the new method. △ Less

Submitted 31 October, 2020; v1 submitted 12 May, 2020; originally announced May 2020.

Asymptotic Properties of High-Dimensional Random Forests

Authors: Chien-Ming Chi, Patrick Vossler, Yingying Fan, Jinchi Lv

Abstract: As a flexible nonparametric learning tool, the random forests algorithm has been widely applied to various real applications with appealing empirical performance, even in the presence of high-dimensional feature space. Unveiling the underlying mechanisms has led to some important recent theoretical results on the consistency of the random forests algorithm and its variants. However, to our knowled… ▽ More As a flexible nonparametric learning tool, the random forests algorithm has been widely applied to various real applications with appealing empirical performance, even in the presence of high-dimensional feature space. Unveiling the underlying mechanisms has led to some important recent theoretical results on the consistency of the random forests algorithm and its variants. However, to our knowledge, almost all existing works concerning random forests consistency in high dimensional setting were established for various modified random forests models where the splitting rules are independent of the response; a few exceptions assume simple data generating models with binary features. In light of this, in this paper we derive the consistency rates for the random forests algorithm associated with the sample CART splitting criterion, which is the one used in the original version of the algorithm, in a general high-dimensional nonparametric regression setting through a bias-variance decomposition analysis. Our new theoretical results show that random forests can indeed adapt to high dimensionality and allow for discontinuous regression function. Our bias analysis characterizes explicitly how the random forests bias depends on the sample size, tree height, and column subsampling parameter. Some limitations on our current results are also discussed. △ Less

Submitted 24 September, 2022; v1 submitted 29 April, 2020; originally announced April 2020.

Comments: 64 pages, 5 figures, to appear in The Annals of Statistics

A Visual-inertial Navigation Method for High-Speed Unmanned Aerial Vehicles

Authors: Xin-long Luo, Jia-hui Lv, Geng Sun

Abstract: This paper investigates the localization problem of high-speed high-altitude unmanned aerial vehicle (UAV) with a monocular camera and inertial navigation system. It proposes a navigation method utilizing the complementarity of vision and inertial devices to overcome the singularity which arises from the horizontal flight of UAV. Furthermore, it modifies the mathematical model of localization prob… ▽ More This paper investigates the localization problem of high-speed high-altitude unmanned aerial vehicle (UAV) with a monocular camera and inertial navigation system. It proposes a navigation method utilizing the complementarity of vision and inertial devices to overcome the singularity which arises from the horizontal flight of UAV. Furthermore, it modifies the mathematical model of localization problem via separating linear parts from nonlinear parts and replaces a nonlinear least-squares problem with a linearly equality-constrained optimization problem. In order to avoid the ill-condition property near the optimal point of sequential unconstrained minimization techniques(penalty methods), it constructs a semi-implicit continuous method with a trust-region technique based on a differential-algebraic dynamical system to solve the linearly equality-constrained optimization problem. It also analyzes the global convergence property of the semi-implicit continuous method in an infinity integrated interval other than the traditional convergence analysis of numerical methods for ordinary differential equations in a finite integrated interval. Finally, the promising numerical results are also presented. △ Less

Submitted 11 February, 2020; originally announced February 2020.

MSC Class: 65H17; 65J15; 65K05; 65L05

New superconvergent structures developed from the finite volume element method in 1D

Authors: Xiang Wang, Junliang Lv, Yonghai Li

Abstract: New superconvergent structures are introduced by the finite volume element method (FVEM), which allow us to choose the superconvergent points freely. The general orthogonal condition and the modified M-decomposition (MMD) technique are established to prove the superconvergence properties of the new structures. In addition, the relationships between the orthogonal condition and the convergence prop… ▽ More New superconvergent structures are introduced by the finite volume element method (FVEM), which allow us to choose the superconvergent points freely. The general orthogonal condition and the modified M-decomposition (MMD) technique are established to prove the superconvergence properties of the new structures. In addition, the relationships between the orthogonal condition and the convergence properties for the FVE schemes are carried out in Table 1. Numerical results are given to illustrate the theoretical results. △ Less

Submitted 10 February, 2020; originally announced February 2020.

MSC Class: 65N12; 65N08; 65N30

Journal ref: Mathematics of Computation, 90 (2021): 1179-1205

Asymptotic Distributions of High-Dimensional Distance Correlation Inference

Authors: Lan Gao, Yingying Fan, Jinchi Lv, Qi-Man Shao

Abstract: Distance correlation has become an increasingly popular tool for detecting the nonlinear dependence between a pair of potentially high-dimensional random vectors. Most existing works have explored its asymptotic distributions under the null hypothesis of independence between the two random vectors when only the sample size or the dimensionality diverges. Yet its asymptotic null distribution for th… ▽ More Distance correlation has become an increasingly popular tool for detecting the nonlinear dependence between a pair of potentially high-dimensional random vectors. Most existing works have explored its asymptotic distributions under the null hypothesis of independence between the two random vectors when only the sample size or the dimensionality diverges. Yet its asymptotic null distribution for the more realistic setting when both sample size and dimensionality diverge in the full range remains largely underdeveloped. In this paper, we fill such a gap and develop central limit theorems and associated rates of convergence for a rescaled test statistic based on the bias-corrected distance correlation in high dimensions under some mild regularity conditions and the null hypothesis. Our new theoretical results reveal an interesting phenomenon of blessing of dimensionality for high-dimensional distance correlation inference in the sense that the accuracy of normal approximation can increase with dimensionality. Moreover, we provide a general theory on the power analysis under the alternative hypothesis of dependence, and further justify the capability of the rescaled distance correlation in capturing the pure nonlinear dependency under moderately high dimensionality for a certain type of alternative hypothesis. The theoretical results and finite-sample performance of the rescaled statistic are illustrated with several simulation examples and a blockchain application. △ Less

Submitted 20 October, 2020; v1 submitted 28 October, 2019; originally announced October 2019.

Comments: 67 pages

Journal ref: Ann. Statist. 49(4): 1999-2020 (August 2021)

SIMPLE: Statistical Inference on Membership Profiles in Large Networks

Authors: Jianqing Fan, Yingying Fan, Xiao Han, Jinchi Lv

Abstract: Network data is prevalent in many contemporary big data applications in which a common interest is to unveil important latent links between different pairs of nodes. Yet a simple fundamental question of how to precisely quantify the statistical uncertainty associated with the identification of latent links still remains largely unexplored. In this paper, we propose the method of statistical infere… ▽ More Network data is prevalent in many contemporary big data applications in which a common interest is to unveil important latent links between different pairs of nodes. Yet a simple fundamental question of how to precisely quantify the statistical uncertainty associated with the identification of latent links still remains largely unexplored. In this paper, we propose the method of statistical inference on membership profiles in large networks (SIMPLE) in the setting of degree-corrected mixed membership model, where the null hypothesis assumes that the pair of nodes share the same profile of community memberships. In the simpler case of no degree heterogeneity, the model reduces to the mixed membership model for which an alternative more robust test is also proposed. Both tests are of the Hotelling-type statistics based on the rows of empirical eigenvectors or their ratios, whose asymptotic covariance matrices are very challenging to derive and estimate. Nevertheless, their analytical expressions are unveiled and the unknown covariance matrices are consistently estimated. Under some mild regularity conditions, we establish the exact limiting distributions of the two forms of SIMPLE test statistics under the null hypothesis and contiguous alternative hypothesis. They are the chi-square distributions and the noncentral chi-square distributions, respectively, with degrees of freedom depending on whether the degrees are corrected or not. We also address the important issue of estimating the unknown number of communities and establish the asymptotic properties of the associated test statistics. The advantages and practical utility of our new procedures in terms of both size and power are demonstrated through several simulation examples and real network applications. △ Less

Submitted 29 August, 2021; v1 submitted 3 October, 2019; originally announced October 2019.

Comments: 59 pages, 4 figures; Journal of the Royal Statistical Society Series B, to appear

Asymptotic Theory of Eigenvectors for Random Matrices with Diverging Spikes

Authors: Jianqing Fan, Yingying Fan, Xiao Han, Jinchi Lv

Abstract: Characterizing the asymptotic distributions of eigenvectors for large random matrices poses important challenges yet can provide useful insights into a range of statistical applications. To this end, in this paper we introduce a general framework of asymptotic theory of eigenvectors (ATE) for large spiked random matrices with diverging spikes and heterogeneous variances, and establish the asymptot… ▽ More Characterizing the asymptotic distributions of eigenvectors for large random matrices poses important challenges yet can provide useful insights into a range of statistical applications. To this end, in this paper we introduce a general framework of asymptotic theory of eigenvectors (ATE) for large spiked random matrices with diverging spikes and heterogeneous variances, and establish the asymptotic properties of the spiked eigenvectors and eigenvalues for the scenario of the generalized Wigner matrix noise. Under some mild regularity conditions, we provide the asymptotic expansions for the spiked eigenvalues and show that they are asymptotically normal after some normalization. For the spiked eigenvectors, we establish asymptotic expansions for the general linear combination and further show that it is asymptotically normal after some normalization, where the weight vector can be arbitrary. We also provide a more general asymptotic theory for the spiked eigenvectors using the bilinear form. Simulation studies verify the validity of our new theoretical results. Our family of models encompasses many popularly used ones such as the stochastic block models with or without overlapping communities for network analysis and the topic models for text analysis, and our general theory can be exploited for statistical inference in these large-scale applications. △ Less

Submitted 13 October, 2020; v1 submitted 18 February, 2019; originally announced February 2019.

Comments: 78 pages, 3 figures; Journal of the American Statistical Association, to appear

An Adaptive Finite Element DtN Method for Maxwell's Equations in Biperiodic Structures

Authors: Xue Jiang, Peijun Li, Junliang Lv, Zhoufeng Wang, Haijun Wu, Weiying Zheng

Abstract: Consider the diffraction of an electromagnetic plane wave by a biperiodic structure where the wave propagation is governed by the three-dimensional Maxwell equations. Based on transparent boundary condition, the grating problem is formulated into a boundary value problem in a bounded domain. Using a duality argument technique, we derive an a posteriori error estimate for the finite element method… ▽ More Consider the diffraction of an electromagnetic plane wave by a biperiodic structure where the wave propagation is governed by the three-dimensional Maxwell equations. Based on transparent boundary condition, the grating problem is formulated into a boundary value problem in a bounded domain. Using a duality argument technique, we derive an a posteriori error estimate for the finite element method with the truncation of the nonlocal Dirichlet-to-Neumann (DtN) boundary operator. The a posteriori error consists of both the finite element approximation error and the truncation error of boundary operator which decays exponentially with respect to the truncation parameter. An adaptive finite element algorithm is developed with error controlled by the a posterior error estimate, which determines the truncation parameter through the truncation error and adjusts the mesh through the finite element approximation error. Numerical experiments are presented to demonstrate the competitive behavior of the proposed adaptive method. △ Less

Submitted 29 November, 2018; originally announced November 2018.

IPAD: Stable Interpretable Forecasting with Knockoffs Inference

Authors: Yingying Fan, Jinchi Lv, Mahrad Sharifvaghefi, Yoshimasa Uematsu

Abstract: Interpretability and stability are two important features that are desired in many contemporary big data applications arising in economics and finance. While the former is enjoyed to some extent by many existing forecasting approaches, the latter in the sense of controlling the fraction of wrongly discovered features which can enhance greatly the interpretability is still largely underdeveloped in… ▽ More Interpretability and stability are two important features that are desired in many contemporary big data applications arising in economics and finance. While the former is enjoyed to some extent by many existing forecasting approaches, the latter in the sense of controlling the fraction of wrongly discovered features which can enhance greatly the interpretability is still largely underdeveloped in the econometric settings. To this end, in this paper we exploit the general framework of model-X knockoffs introduced recently in Candès, Fan, Janson and Lv (2018), which is nonconventional for reproducible large-scale inference in that the framework is completely free of the use of p-values for significance testing, and suggest a new method of intertwined probabilistic factors decoupling (IPAD) for stable interpretable forecasting with knockoffs inference in high-dimensional models. The recipe of the method is constructing the knockoff variables by assuming a latent factor model that is exploited widely in economics and finance for the association structure of covariates. Our method and work are distinct from the existing literature in that we estimate the covariate distribution from data instead of assuming that it is known when constructing the knockoff variables, our procedure does not require any sample splitting, we provide theoretical justifications on the asymptotic false discovery rate control, and the theory for the power analysis is also established. Several simulation examples and the real data analysis further demonstrate that the newly suggested method has appealing finite-sample performance with desired interpretability and stability compared to some popularly used forecasting methods. △ Less

Submitted 6 September, 2018; originally announced September 2018.

Large-Scale Model Selection with Misspecification

Authors: Emre Demirkaya, Yang Feng, Pallavi Basu, Jinchi Lv

Abstract: Model selection is crucial to high-dimensional learning and inference for contemporary big data applications in pinpointing the best set of covariates among a sequence of candidate interpretable models. Most existing work assumes implicitly that the models are correctly specified or have fixed dimensionality. Yet both features of model misspecification and high dimensionality are prevalent in prac… ▽ More Model selection is crucial to high-dimensional learning and inference for contemporary big data applications in pinpointing the best set of covariates among a sequence of candidate interpretable models. Most existing work assumes implicitly that the models are correctly specified or have fixed dimensionality. Yet both features of model misspecification and high dimensionality are prevalent in practice. In this paper, we exploit the framework of model selection principles in misspecified models originated in Lv and Liu (2014) and investigate the asymptotic expansion of Bayesian principle of model selection in the setting of high-dimensional misspecified models. With a natural choice of prior probabilities that encourages interpretability and incorporates Kullback-Leibler divergence, we suggest the high-dimensional generalized Bayesian information criterion with prior probability (HGBIC_p) for large-scale model selection with misspecification. Our new information criterion characterizes the impacts of both model misspecification and high dimensionality on model selection. We further establish the consistency of covariance contrast matrix estimation and the model selection consistency of HGBIC_p in ultra-high dimensions under some mild regularity conditions. The advantages of our new method are supported by numerical studies. △ Less

Submitted 16 March, 2018; originally announced March 2018.

Comments: 38 pages, 2 figures. arXiv admin note: text overlap with arXiv:1412.7468

Nonsparse learning with latent variables

Authors: Zemin Zheng, Jinchi Lv, Wei Lin

Abstract: As a popular tool for producing meaningful and interpretable models, large-scale sparse learning works efficiently when the underlying structures are indeed or close to sparse. However, naively applying the existing regularization methods can result in misleading outcomes due to model misspecification. In particular, the direct sparsity assumption on coefficient vectors has been questioned in real… ▽ More As a popular tool for producing meaningful and interpretable models, large-scale sparse learning works efficiently when the underlying structures are indeed or close to sparse. However, naively applying the existing regularization methods can result in misleading outcomes due to model misspecification. In particular, the direct sparsity assumption on coefficient vectors has been questioned in real applications. Therefore, we consider nonsparse learning with the conditional sparsity structure that the coefficient vector becomes sparse after taking out the impacts of certain unobservable latent variables. A new methodology of nonsparse learning with latent variables (NSL) is proposed to simultaneously recover the significant observable predictors and latent factors as well as their effects. We explore a common latent family incorporating population principal components and derive the convergence rates of both sample principal components and their score vectors that hold for a wide class of distributions. With the properly estimated latent variables, properties including model selection consistency and oracle inequalities under various prediction and estimation losses are established for the proposed methodology. Our new methodology and results are evidenced by simulation and real data examples. △ Less

Submitted 7 October, 2017; originally announced October 2017.

Comments: 30 pages

MSC Class: 62J

RANK: Large-Scale Inference with Graphical Nonlinear Knockoffs

Authors: Yingying Fan, Emre Demirkaya, Gaorong Li, Jinchi Lv

Abstract: Power and reproducibility are key to enabling refined scientific discoveries in contemporary big data applications with general high-dimensional nonlinear models. In this paper, we provide theoretical foundations on the power and robustness for the model-free knockoffs procedure introduced recently in Candès, Fan, Janson and Lv (2016) in high-dimensional setting when the covariate distribution is… ▽ More Power and reproducibility are key to enabling refined scientific discoveries in contemporary big data applications with general high-dimensional nonlinear models. In this paper, we provide theoretical foundations on the power and robustness for the model-free knockoffs procedure introduced recently in Candès, Fan, Janson and Lv (2016) in high-dimensional setting when the covariate distribution is characterized by Gaussian graphical model. We establish that under mild regularity conditions, the power of the oracle knockoffs procedure with known covariate distribution in high-dimensional linear models is asymptotically one as sample size goes to infinity. When moving away from the ideal case, we suggest the modified model-free knockoffs method called graphical nonlinear knockoffs (RANK) to accommodate the unknown covariate distribution. We provide theoretical justifications on the robustness of our modified procedure by showing that the false discovery rate (FDR) is asymptotically controlled at the target level and the power is asymptotically one with the estimated covariate distribution. To the best of our knowledge, this is the first formal theoretical result on the power for the knockoffs procedure. Simulation results demonstrate that compared to existing approaches, our method performs competitively in both FDR control and power. A real data set is analyzed to further assess the performance of the suggested knockoffs procedure. △ Less

Submitted 31 August, 2017; originally announced September 2017.

Comments: 37 pages, 6 tables, 9 pages supplementary material

A note on the bijectivity of antipode of a Hopf algebra and its applications

Authors: Jiafeng Lv, Sei-Qwon Oh, Xingting Wang, Xiaolan Yu

Abstract: Certain sufficient homological and ring-theoretical conditions are given for a Hopf algebra to have bijective antipode with applications to noetherian Hopf algebras regarding their homological behaviors. Certain sufficient homological and ring-theoretical conditions are given for a Hopf algebra to have bijective antipode with applications to noetherian Hopf algebras regarding their homological behaviors. △ Less

Submitted 27 February, 2019; v1 submitted 19 April, 2017; originally announced April 2017.

Comments: 14 pages

MSC Class: 16E65; 16W30; 16W35

Journal ref: Proc. Amer. Math. Soc. 146, no. 11 (2018), 4619-4631

Convergence of the PML solution for elastic wave scattering by biperiodic structures

Authors: Xue Jiang, Peijun Li, Junliang Lv, Weiying Zheng

Abstract: This paper is concerned with the analysis of elastic wave scattering of a time-harmonic plane wave by a biperiodic rigid surface, where the wave propagation is governed by the three-dimensional Navier equation. An exact transparent boundary condition is developed to reduce the scattering problem equivalently into a boundary value problem in a bounded domain. The perfectly matched layer (PML) techn… ▽ More This paper is concerned with the analysis of elastic wave scattering of a time-harmonic plane wave by a biperiodic rigid surface, where the wave propagation is governed by the three-dimensional Navier equation. An exact transparent boundary condition is developed to reduce the scattering problem equivalently into a boundary value problem in a bounded domain. The perfectly matched layer (PML) technique is adopted to truncate the unbounded physical domain into a bounded computational domain. The well-posedness and exponential convergence of the solution are established for the truncated PML problem by developing a PML equivalent transparent boundary condition. The proofs rely on a careful study of the error between the two transparent boundary operators. The work significantly extend the results from the one-dimensional periodic structures to the two-dimensional biperiodic structures. Numerical experiments are included to demonstrate the competitive behavior of the proposed method. △ Less

Submitted 17 November, 2016; originally announced November 2016.

Comments: arXiv admin note: substantial text overlap with arXiv:1605.08746

Panning for Gold: Model-X Knockoffs for High-dimensional Controlled Variable Selection

Authors: Emmanuel Candes, Yingying Fan, Lucas Janson, Jinchi Lv

Abstract: Many contemporary large-scale applications involve building interpretable models linking a large set of potential covariates to a response in a nonlinear fashion, such as when the response is binary. Although this modeling problem has been extensively studied, it remains unclear how to effectively control the fraction of false discoveries even in high-dimensional logistic regression, not to mentio… ▽ More Many contemporary large-scale applications involve building interpretable models linking a large set of potential covariates to a response in a nonlinear fashion, such as when the response is binary. Although this modeling problem has been extensively studied, it remains unclear how to effectively control the fraction of false discoveries even in high-dimensional logistic regression, not to mention general high-dimensional nonlinear models. To address such a practical problem, we propose a new framework of $model$-$X$ knockoffs, which reads from a different perspective the knockoff procedure (Barber and Candès, 2015) originally designed for controlling the false discovery rate in linear models. Whereas the knockoffs procedure is constrained to homoscedastic linear models with $n\ge p$, the key innovation here is that model-X knockoffs provide valid inference from finite samples in settings in which the conditional distribution of the response is arbitrary and completely unknown. Furthermore, this holds no matter the number of covariates. Correct inference in such a broad setting is achieved by constructing knockoff variables probabilistically instead of geometrically. To do this, our approach requires the covariates be random (independent and identically distributed rows) with a distribution that is known, although we provide preliminary experimental evidence that our procedure is robust to unknown/estimated distributions. To our knowledge, no other procedure solves the $controlled$ variable selection problem in such generality, but in the restricted settings where competitors exist, we demonstrate the superior power of knockoffs through simulations. Finally, we apply our procedure to data from a case-control study of Crohn's disease in the United Kingdom, making twice as many discoveries as the original analysis of the same data. △ Less

Submitted 12 December, 2017; v1 submitted 7 October, 2016; originally announced October 2016.

Comments: 39 pages, 10 figures, 2 tables

Tuning-Free Heterogeneity Pursuit in Massive Networks

Authors: Zhao Ren, Yongjian Kang, Yingying Fan, Jinchi Lv

Abstract: Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we exploit multiple networks with Gaussian graphs to encode the connectivity patterns o… ▽ More Heterogeneity is often natural in many contemporary applications involving massive data. While posing new challenges to effective learning, it can play a crucial role in powering meaningful scientific discoveries through the understanding of important differences among subpopulations of interest. In this paper, we exploit multiple networks with Gaussian graphs to encode the connectivity patterns of a large number of features on the subpopulations. To uncover the heterogeneity of these structures across subpopulations, we suggest a new framework of tuning-free heterogeneity pursuit (THP) via large-scale inference, where the number of networks is allowed to diverge. In particular, two new tests, the chi-based test and the linear functional-based test, are introduced and their asymptotic null distributions are established. Under mild regularity conditions, we establish that both tests are optimal in achieving the testable region boundary and the sample size requirement for the latter test is minimal. Both theoretical guarantees and the tuning-free feature stem from efficient multiple-network estimation by our newly suggested approach of heterogeneous group square-root Lasso (HGSL) for high-dimensional multi-response regression with heterogeneous noises. To solve this convex program, we further introduce a tuning-free algorithm that is scalable and enjoys provable convergence to the global optimum. Both computational and theoretical advantages of our procedure are elucidated through simulation and real data examples. △ Less

Submitted 12 June, 2016; originally announced June 2016.

Comments: 29 pages for the main text including 1 figure and 7 tables, 28 pages for the Supplementary Material

An adaptive finite element PML method for the elastic wave scattering problem in periodic structures

Authors: Xue Jiang, Peijun Li, Junliang Lv, Weiying Zheng

Abstract: An adaptive finite element method is presented for the elastic scattering of a time-harmonic plane wave by a periodic surface. First, the unbounded physical domain is truncated into a bounded computational domain by introducing the perfectly matched layer (PML) technique. The well-posedness and exponential convergence of the solution are established for the truncated PML problem by developing an e… ▽ More An adaptive finite element method is presented for the elastic scattering of a time-harmonic plane wave by a periodic surface. First, the unbounded physical domain is truncated into a bounded computational domain by introducing the perfectly matched layer (PML) technique. The well-posedness and exponential convergence of the solution are established for the truncated PML problem by developing an equivalent transparent boundary condition. Second, an a posteriori error estimate is deduced for the discrete problem and is used to determine the finite elements for refinements and to determine the PML parameters. Numerical experiments are included to demonstrate the competitive behavior of the proposed adaptive method. △ Less

Submitted 27 May, 2016; originally announced May 2016.

Asymptotic equivalence of regularization methods in thresholded parameter space

Authors: Yingying Fan, Jinchi Lv

Abstract: High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this paper, we characterize the asymptotic equivalence of regularization methods, with ge… ▽ More High-dimensional data analysis has motivated a spectrum of regularization methods for variable selection and sparse modeling, with two popular classes of convex ones and concave ones. A long debate has been on whether one class dominates the other, an important question both in theory and to practitioners. In this paper, we characterize the asymptotic equivalence of regularization methods, with general penalty functions, in a thresholded parameter space under the generalized linear model setting, where the dimensionality can grow up to exponentially with the sample size. To assess their performance, we establish the oracle inequalities, as in Bickel, Ritov and Tsybakov (2009), of the global minimizer for these methods under various prediction and variable selection losses. These results reveal an interesting phase transition phenomenon. For polynomially growing dimensionality, the $L_1$-regularization method of Lasso and concave methods are asymptotically equivalent, having the same convergence rates in the oracle inequalities. For exponentially growing dimensionality, concave methods are asymptotically equivalent but have faster convergence rates than the Lasso. We also establish a stronger property of the oracle risk inequalities of the regularization methods, as well as the sampling properties of computable solutions. Our new theoretical results are illustrated and justified by simulation and real data examples. △ Less

Submitted 11 May, 2016; originally announced May 2016.

Comments: 39 pages, 3 figures

MSC Class: 62J07(Primary) 62F07(Secondary)

Journal ref: Journal of the American Statistical Association 108, 1044-1061

Model Selection in High-Dimensional Misspecified Models

Authors: Pallavi Basu, Yang Feng, Jinchi Lv

Abstract: Model selection is indispensable to high-dimensional sparse modeling in selecting the best set of covariates among a sequence of candidate models. Most existing work assumes implicitly that the model is correctly specified or of fixed dimensions. Yet model misspecification and high dimensionality are common in real applications. In this paper, we investigate two classical Kullback-Leibler divergen… ▽ More Model selection is indispensable to high-dimensional sparse modeling in selecting the best set of covariates among a sequence of candidate models. Most existing work assumes implicitly that the model is correctly specified or of fixed dimensions. Yet model misspecification and high dimensionality are common in real applications. In this paper, we investigate two classical Kullback-Leibler divergence and Bayesian principles of model selection in the setting of high-dimensional misspecified models. Asymptotic expansions of these principles reveal that the effect of model misspecification is crucial and should be taken into account, leading to the generalized AIC and generalized BIC in high dimensions. With a natural choice of prior probabilities, we suggest the generalized BIC with prior probability which involves a logarithmic factor of the dimensionality in penalizing model complexity. We further establish the consistency of the covariance contrast matrix estimator in a general setting. Our results and new method are supported by numerical studies. △ Less

Submitted 23 December, 2014; originally announced December 2014.

Comments: 43 pages

Discussion: "A significance test for the lasso"

Authors: Jinchi Lv, Zemin Zheng

Abstract: Discussion of "A significance test for the lasso" by Richard Lockhart, Jonathan Taylor, Ryan J. Tibshirani, Robert Tibshirani [arXiv:1301.7161]. Discussion of "A significance test for the lasso" by Richard Lockhart, Jonathan Taylor, Ryan J. Tibshirani, Robert Tibshirani [arXiv:1301.7161]. △ Less

Submitted 27 May, 2014; originally announced May 2014.

Comments: Published in at http://dx.doi.org/10.1214/13-AOS1175D the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1175D

Journal ref: Annals of Statistics 2014, Vol. 42, No. 2, 493-500

Impacts of high dimensionality in finite samples

Authors: Jinchi Lv

Abstract: High-dimensional data sets are commonly collected in many contemporary applications arising in various fields of scientific research. We present two views of finite samples in high dimensions: a probabilistic one and a nonprobabilistic one. With the probabilistic view, we establish the concentration property and robust spark bound for large random design matrix generated from elliptical distributi… ▽ More High-dimensional data sets are commonly collected in many contemporary applications arising in various fields of scientific research. We present two views of finite samples in high dimensions: a probabilistic one and a nonprobabilistic one. With the probabilistic view, we establish the concentration property and robust spark bound for large random design matrix generated from elliptical distributions, with the former related to the sure screening property and the latter related to sparse model identifiability. An interesting concentration phenomenon in high dimensions is revealed. With the nonprobabilistic view, we derive general bounds on dimensionality with some distance constraint on sparse models. These results provide new insights into the impacts of high dimensionality in finite samples. △ Less

Submitted 12 November, 2013; originally announced November 2013.

Comments: Published in at http://dx.doi.org/10.1214/13-AOS1149 the Annals of Statistics (http://www.imstat.org/aos/) by the Institute of Mathematical Statistics (http://www.imstat.org)

Report number: IMS-AOS-AOS1149

Journal ref: Annals of Statistics 2013, Vol. 41, No. 4, 2236-2262

High-Dimensional Sparse Additive Hazards Regression

Authors: Wei Lin, Jinchi Lv

Abstract: High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by modern applications in high-throughput genomic data analysis and credit risk analysis. In this article, we propose a class of regularization methods for simultaneous variable selection and estimation in the additive hazards model, by combining the nonconcave penalized likelihood approac… ▽ More High-dimensional sparse modeling with censored survival data is of great practical importance, as exemplified by modern applications in high-throughput genomic data analysis and credit risk analysis. In this article, we propose a class of regularization methods for simultaneous variable selection and estimation in the additive hazards model, by combining the nonconcave penalized likelihood approach and the pseudoscore method. In a high-dimensional setting where the dimensionality can grow fast, polynomially or nonpolynomially, with the sample size, we establish the weak oracle property and oracle property under mild, interpretable conditions, thus providing strong performance guarantees for the proposed methodology. Moreover, we show that the regularity conditions required by the $L_1$ method are substantially relaxed by a certain class of sparsity-inducing concave penalties. As a result, concave penalties such as the smoothly clipped absolute deviation (SCAD), minimax concave penalty (MCP), and smooth integration of counting and absolute deviation (SICA) can significantly improve on the $L_1$ method and yield sparser models with better prediction performance. We present a coordinate descent algorithm for efficient implementation and rigorously investigate its convergence properties. The practical utility and effectiveness of the proposed methods are demonstrated by simulation studies and a real data example. △ Less

Submitted 26 December, 2012; originally announced December 2012.

Comments: 41 pages, 3 figures, to appear in Journal of the American Statistical Association (http://www.tandfonline.com/r/JASA)

Journal ref: Journal of the American Statistical Association (2013), 108, 247-264

Model Selection Principles in Misspecified Models

Authors: Jinchi Lv, Jun S. Liu

Abstract: Model selection is of fundamental importance to high dimensional modeling featured in many contemporary applications. Classical principles of model selection include the Kullback-Leibler divergence principle and the Bayesian principle, which lead to the Akaike information criterion and Bayesian information criterion when models are correctly specified. Yet model misspecification is unavoidable whe… ▽ More Model selection is of fundamental importance to high dimensional modeling featured in many contemporary applications. Classical principles of model selection include the Kullback-Leibler divergence principle and the Bayesian principle, which lead to the Akaike information criterion and Bayesian information criterion when models are correctly specified. Yet model misspecification is unavoidable when we have no knowledge of the true model or when we have the correct family of distributions but miss some true predictor. In this paper, we propose a family of semi-Bayesian principles for model selection in misspecified models, which combine the strengths of the two well-known principles. We derive asymptotic expansions of the semi-Bayesian principles in misspecified generalized linear models, which give the new semi-Bayesian information criteria (SIC). A specific form of SIC admits a natural decomposition into the negative maximum quasi-log-likelihood, a penalty on model dimensionality, and a penalty on model misspecification directly. Numerical studies demonstrate the advantage of the newly proposed SIC methodology for model selection in both correctly specified and misspecified models. △ Less

Submitted 11 May, 2016; v1 submitted 29 May, 2010; originally announced May 2010.

Comments: 25 pages, 6 tables

MSC Class: 62J12(Primary); 62B10; 62F07; 62F15; 62J07(Secondary)

Journal ref: Journal of the Royal Statistical Society Series B 76, 141-167 (2014)

A Selective Overview of Variable Selection in High Dimensional Feature Space (Invited Review Article)

Authors: Jianqing Fan, Jinchi Lv

Abstract: High dimensional statistical problems arise from diverse fields of scientific research and technological development. Variable selection plays a pivotal role in contemporary statistical learning and scientific discoveries. The traditional idea of best subset selection methods, which can be regarded as a specific form of penalized likelihood, is computationally too expensive for many modern stati… ▽ More High dimensional statistical problems arise from diverse fields of scientific research and technological development. Variable selection plays a pivotal role in contemporary statistical learning and scientific discoveries. The traditional idea of best subset selection methods, which can be regarded as a specific form of penalized likelihood, is computationally too expensive for many modern statistical applications. Other forms of penalized likelihood methods have been successfully developed over the last decade to cope with high dimensionality. They have been widely applied for simultaneously selecting important variables and estimating their effects in high dimensional statistical inference. In this article, we present a brief account of the recent developments of theory, methods, and implementations for high dimensional variable selection. What limits of the dimensionality such methods can handle, what the role of penalty functions is, and what the statistical properties are rapidly drive the advances of the field. The properties of non-concave penalized likelihood and its roles in high dimensional statistical modeling are emphasized. We also review some recent advances in ultra-high dimensional variable selection, with emphasis on independence screening and two-scale methods. △ Less

Submitted 6 October, 2009; originally announced October 2009.

Comments: To appear in Statistica Sinica, 44 pages, 5 figures

MSC Class: Primary 62J99; secondary 62F12; 68Q32

Non-Concave Penalized Likelihood with NP-Dimensionality

Authors: Jianqing Fan, Jinchi Lv

Abstract: Penalized likelihood methods are fundamental to ultra-high dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of Non-Polynomial (NP) order of sample size, for a class of penal… ▽ More Penalized likelihood methods are fundamental to ultra-high dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of Non-Polynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the $L_1$-penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis. △ Less

Submitted 6 October, 2009; originally announced October 2009.

Comments: 37 pages, 2 figures

MSC Class: Primary 62F12; secondary 62J12