-
Estimation and variable selection in nonlinear mixed-effects models
Authors:
Antoine Caillebotte,
Estelle Kuhn,
Sarah Lemler
Abstract:
We consider nonlinear mixed effects models including high-dimensional covariates to model individual parameters. The objective is to identify relevant covariates and estimate model parameters. We combine a penalized LASSO-type estimator with an eBIC model choice criterion to select the covariates of interest. Then we estimate the parameters by maximum likelihood in the reduced model. We calculate…
▽ More
We consider nonlinear mixed effects models including high-dimensional covariates to model individual parameters. The objective is to identify relevant covariates and estimate model parameters. We combine a penalized LASSO-type estimator with an eBIC model choice criterion to select the covariates of interest. Then we estimate the parameters by maximum likelihood in the reduced model. We calculate the LASSO-type penalized estimator by a weighted proximal gradient descent algorithm with an adaptive learning rate. This choice allows us in particular to consider models that do not necessarily belong to the curved exponential family. We compare first the performance of the proposed methodology with those of the glmmLasso procedure in a linear mixed effects model in a simulation study. We then illustrate its performance in a nonlinear mixed-effects logistic growth model through simulation.
△ Less
Submitted 26 March, 2025;
originally announced March 2025.
-
A Unified Bayesian Perspective for Conventional and Robust Adaptive Filters
Authors:
Leszek Szczecinski,
Jacob Benesty,
Eduardo Vinicius Kuhn
Abstract:
In this work, we present a new perspective on the origin and interpretation of adaptive filters. By applying Bayesian principles of recursive inference from the state-space model and using a series of simplifications regarding the structure of the solution, we can present, in a unified framework, derivations of many adaptive filters that depend on the probabilistic model of the measurement noise.…
▽ More
In this work, we present a new perspective on the origin and interpretation of adaptive filters. By applying Bayesian principles of recursive inference from the state-space model and using a series of simplifications regarding the structure of the solution, we can present, in a unified framework, derivations of many adaptive filters that depend on the probabilistic model of the measurement noise. In particular, under a Gaussian model, we obtain solutions well-known in the literature (such as LMS, NLMS, or Kalman filter), while using non-Gaussian noise, we derive new adaptive algorithms. Notably, under the assumption of Laplacian noise, we obtain a family of robust filters of which the sign-error algorithm is a well-known member, while other algorithms, derived effortlessly in the proposed framework, are entirely new. Numerical examples are shown to illustrate the properties and provide a better insight into the performance of the derived adaptive filters.
△ Less
Submitted 1 July, 2025; v1 submitted 25 February, 2025;
originally announced February 2025.
-
Estimation and variable selection in a joint model of survival times and longitudinal outcomes with random effects
Authors:
Antoine Caillebotte,
Estelle Kuhn,
Sarah Lemler
Abstract:
This paper considers a joint survival and mixed-effects model to explain the survival time from longitudinal data and high-dimensional covariates. The longitudinal data is modeled using a nonlinear effects model, where the regression function serves as a link function incorporated into a Cox model as a covariate. In that way, the longitudinal data is related to the survival time at a given time. A…
▽ More
This paper considers a joint survival and mixed-effects model to explain the survival time from longitudinal data and high-dimensional covariates. The longitudinal data is modeled using a nonlinear effects model, where the regression function serves as a link function incorporated into a Cox model as a covariate. In that way, the longitudinal data is related to the survival time at a given time. Additionally, the Cox model takes into account the inclusion of high-dimensional covariates. The main objectives of this research are two-fold: first, to identify the relevant covariates that contribute to explaining survival time, and second, to estimate all unknown parameters of the joint model. For that purpose, we consider the maximization of a Lasso penalized likelihood. To tackle the optimization problem, we implement a pre-conditioned stochastic gradient to handle the latent variables of the nonlinear mixed-effects model associated with a proximal operator to manage the non-differentiability of the penalty. We provide relevant simulations that showcase the performance of the proposed variable selection and parameters' estimation method in the joint modeling of a Cox and logistic model.
△ Less
Submitted 22 July, 2024; v1 submitted 29 June, 2023;
originally announced June 2023.
-
Efficient preconditioned stochastic gradient descent for estimation in latent variable models
Authors:
Charlotte Baey,
Maud Delattre,
Estelle Kuhn,
Jean-Benoist Leger,
Sarah Lemler
Abstract:
Latent variable models are powerful tools for modeling complex phenomena involving in particular partially observed data, unobserved variables or underlying complex unknown structures. Inference is often difficult due to the latent structure of the model. To deal with parameter estimation in the presence of latent variables, well-known efficient methods exist, such as gradient-based and EM-type al…
▽ More
Latent variable models are powerful tools for modeling complex phenomena involving in particular partially observed data, unobserved variables or underlying complex unknown structures. Inference is often difficult due to the latent structure of the model. To deal with parameter estimation in the presence of latent variables, well-known efficient methods exist, such as gradient-based and EM-type algorithms, but with practical and theoretical limitations. In this paper, we propose as an alternative for parameter estimation an efficient preconditioned stochastic gradient algorithm. Our method includes a preconditioning step based on a positive definite Fisher information matrix estimate. We prove convergence results for the proposed algorithm under mild assumptions for very general latent variables models. We illustrate through relevant simulations the performance of the proposed methodology in a nonlinear mixed effects model and in a stochastic block model.
△ Less
Submitted 22 June, 2023;
originally announced June 2023.
-
A New Approach to Inverting Backscatter and Extinction from Photon-Limited Lidar Observations
Authors:
Willem J. Marais,
Robert E. Holz,
Yu Hen Hu,
Ralph E. Kuehn,
Edwin E. Eloranta,
Rebecca M. Willett
Abstract:
Atmospheric lidar observations provide a unique capability to directly observe the vertical column of cloud and aerosol scattering properties. Detector and solar background noise, however, hinder the ability of lidar systems to provide reliable backscatter and extinction cross-section estimates. Standard methods for solving this inverse problem are most effective with high signal-to-noise ratio ob…
▽ More
Atmospheric lidar observations provide a unique capability to directly observe the vertical column of cloud and aerosol scattering properties. Detector and solar background noise, however, hinder the ability of lidar systems to provide reliable backscatter and extinction cross-section estimates. Standard methods for solving this inverse problem are most effective with high signal-to-noise ratio observations that are only available at low-resolution in uniform scenes. This paper describes a novel method for solving the inverse problem with high-resolution, lower signal-to-noise ratio observations that are effective in non-uniform scenes. The novelty is twofold. First, the inference of the backscatter and extinction are done on images, whereas current lidar algorithms only use the information content of single profiles. Hence, the latent spatial and temporal information in the noisy images are utilized to infer the cross-sections. Second, the noise associated with photon-counting lidar observations can be modeled using a Poisson distribution, and state-of-the-art tools for solving Poisson inverse problems are adapted to the atmospheric lidar problem. It is demonstrated through photon-counting High Spectral Resolution Lidar (HSRL) simulations that the proposed algorithm yield inverted backscatter and extinction cross-sections (per unit volume) with smaller Mean Squared Error (MSE) values at higher spatial and temporal resolutions, compared to the standard approach. Two case studies of real experimental data are also provided where the proposed algorithm is applied on HSRL observations and the inverted backscatter and extinction cross-sections are compared against the standard approach.
△ Less
Submitted 20 June, 2016; v1 submitted 7 March, 2016;
originally announced March 2016.
-
Efficiency of the Wang-Landau algorithm: a simple test case
Authors:
G. Fort,
B. Jourdain,
E. Kuhn,
T Lelièvre,
G. Stoltz
Abstract:
We analyze the efficiency of the Wang-Landau algorithm to sample a multimodal distribution on a prototypical simple test case. We show that the exit time from a metastable state is much smaller for the Wang Landau dynamics than for the original standard Metropolis-Hastings algorithm, in some asymptotic regime. Our results are confirmed by numerical experiments on a more realistic test case.
We analyze the efficiency of the Wang-Landau algorithm to sample a multimodal distribution on a prototypical simple test case. We show that the exit time from a metastable state is much smaller for the Wang Landau dynamics than for the original standard Metropolis-Hastings algorithm, in some asymptotic regime. Our results are confirmed by numerical experiments on a more realistic test case.
△ Less
Submitted 8 February, 2014; v1 submitted 24 October, 2013;
originally announced October 2013.
-
Convergence of the Wang-Landau algorithm
Authors:
Gersende Fort,
Benjamin Jourdain,
Estelle Kuhn,
Tony Lelièvre,
Gabriel Stoltz
Abstract:
We analyze the convergence properties of the Wang-Landau algorithm. This sampling method belongs to the general class of adaptive importance sampling strategies which use the free energy along a chosen reaction coordinate as a bias. Such algorithms are very helpful to enhance the sampling properties of Markov Chain Monte Carlo algorithms, when the dynamics is metastable. We prove the convergence o…
▽ More
We analyze the convergence properties of the Wang-Landau algorithm. This sampling method belongs to the general class of adaptive importance sampling strategies which use the free energy along a chosen reaction coordinate as a bias. Such algorithms are very helpful to enhance the sampling properties of Markov Chain Monte Carlo algorithms, when the dynamics is metastable. We prove the convergence of the Wang-Landau algorithm and an associated central limit theorem.
△ Less
Submitted 26 September, 2013; v1 submitted 30 July, 2012;
originally announced July 2012.
-
Convergent Stochastic Expectation Maximization algorithm with efficient sampling in high dimension. Application to deformable template model estimation
Authors:
Stéphanie Allassonniere,
Estelle Kuhn
Abstract:
Estimation in the deformable template model is a big challenge in image analysis. The issue is to estimate an atlas of a population. This atlas contains a template and the corresponding geometrical variability of the observed shapes. The goal is to propose an accurate algorithm with low computational cost and with theoretical guaranties of relevance. This becomes very demanding when dealing with h…
▽ More
Estimation in the deformable template model is a big challenge in image analysis. The issue is to estimate an atlas of a population. This atlas contains a template and the corresponding geometrical variability of the observed shapes. The goal is to propose an accurate algorithm with low computational cost and with theoretical guaranties of relevance. This becomes very demanding when dealing with high dimensional data which is particularly the case of medical images. We propose to use an optimized Monte Carlo Markov Chain method into a stochastic Expectation Maximization algorithm in order to estimate the model parameters by maximizing the likelihood. In this paper, we present a new Anisotropic Metropolis Adjusted Langevin Algorithm which we use as transition in the MCMC method. We first prove that this new sampler leads to a geometrically uniformly ergodic Markov chain. We prove also that under mild conditions, the estimated parameters converge almost surely and are asymptotically Gaussian distributed. The methodology developed is then tested on handwritten digits and some 2D and 3D medical images for the deformable model estimation. More widely, the proposed algorithm can be used for a large range of models in many fields of applications such as pharmacology or genetic.
△ Less
Submitted 6 September, 2013; v1 submitted 25 July, 2012;
originally announced July 2012.
-
Stochastic Algorithm For Parameter Estimation For Dense Deformable Template Mixture Model
Authors:
Stéphanie Allassonnière,
Estelle Kuhn
Abstract:
Estimating probabilistic deformable template models is a new approach in the fields of computer vision and probabilistic atlases in computational anatomy. A first coherent statistical framework modelling the variability as a hidden random variable has been given by Allassonnière, Amit and Trouvé in [1] in simple and mixture of deformable template models. A consistent stochastic algorithm has bee…
▽ More
Estimating probabilistic deformable template models is a new approach in the fields of computer vision and probabilistic atlases in computational anatomy. A first coherent statistical framework modelling the variability as a hidden random variable has been given by Allassonnière, Amit and Trouvé in [1] in simple and mixture of deformable template models. A consistent stochastic algorithm has been introduced in [2] to face the problem encountered in [1] for the convergence of the estimation algorithm for the one component model in the presence of noise. We propose here to go on in this direction of using some "SAEM-like" algorithm to approximate the MAP estimator in the general Bayesian setting of mixture of deformable template model. We also prove the convergence of this algorithm toward a critical point of the penalised likelihood of the observations and illustrate this with handwritten digit images.
△ Less
Submitted 16 January, 2009; v1 submitted 11 February, 2008;
originally announced February 2008.
