Showing 1–9 of 9 results for author: Schumacher, L

Validation and Comparison of Non-Stationary Cognitive Models: A Diffusion Model Application

Authors: Lukas Schumacher, Martin Schnuerch, Andreas Voss, Stefan T. Radev

Abstract: Cognitive processes undergo various fluctuations and transient states across different temporal scales. Superstatistics are emerging as a flexible framework for incorporating such non-stationary dynamics into existing cognitive model classes. In this work, we provide the first experimental validation of superstatistics and formal comparison of four non-stationary diffusion decision models in a spe… ▽ More Cognitive processes undergo various fluctuations and transient states across different temporal scales. Superstatistics are emerging as a flexible framework for incorporating such non-stationary dynamics into existing cognitive model classes. In this work, we provide the first experimental validation of superstatistics and formal comparison of four non-stationary diffusion decision models in a specifically designed perceptual decision-making task. Task difficulty and speed-accuracy trade-off were systematically manipulated to induce expected changes in model parameters. To validate our models, we assess whether the inferred parameter trajectories align with the patterns and sequences of the experimental manipulations. To address computational challenges, we present novel deep learning techniques for amortized Bayesian estimation and comparison of models with time-varying parameters. Our findings indicate that transition models incorporating both gradual and abrupt parameter shifts provide the best fit to the empirical data. Moreover, we find that the inferred parameter trajectories closely mirror the sequence of experimental manipulations. Posterior re-simulations further underscore the ability of the models to faithfully reproduce critical data patterns. Accordingly, our results suggest that the inferred non-stationary dynamics may reflect actual changes in the targeted psychological constructs. We argue that our initial experimental validation paves the way for the widespread application of superstatistics in cognitive modeling and beyond. △ Less

Submitted 1 October, 2024; v1 submitted 7 December, 2023; originally announced January 2024.

The use of the EM algorithm for regularization problems in high-dimensional linear mixed-effects models

Authors: Daniela C. R. Oliveira, Fernanda L. Schumacher, Victor H. Lachos

Abstract: The EM algorithm is a popular tool for maximum likelihood estimation but has not been used much for high-dimensional regularization problems in linear mixed-effects models. In this paper, we introduce the EMLMLasso algorithm, which combines the EM algorithm and the popular and efficient R package glmnet for Lasso variable selection of fixed effects in linear mixed-effects models. We compare the pe… ▽ More The EM algorithm is a popular tool for maximum likelihood estimation but has not been used much for high-dimensional regularization problems in linear mixed-effects models. In this paper, we introduce the EMLMLasso algorithm, which combines the EM algorithm and the popular and efficient R package glmnet for Lasso variable selection of fixed effects in linear mixed-effects models. We compare the performance of our proposed EMLMLasso algorithm with the one implemented in the well-known R package glmmLasso through the analyses of both simulated and real-world applications. The simulations and applications demonstrated good properties, such as consistency, and the effectiveness of the proposed variable selection procedure, for both $p < n$ and $p > n$. Moreover, in all evaluated scenarios, the EMLMLasso algorithm outperformed glmmLasso. The proposed method is quite general and can be easily extended for ridge and elastic net penalties in linear mixed-effects models. △ Less

Submitted 2 August, 2023; originally announced August 2023.

BayesFlow: Amortized Bayesian Workflows With Neural Networks

Authors: Stefan T Radev, Marvin Schmitt, Lukas Schumacher, Lasse Elsemüller, Valentin Pratz, Yannik Schälte, Ullrich Köthe, Paul-Christian Bürkner

Abstract: Modern Bayesian inference involves a mixture of computational techniques for estimating, validating, and drawing conclusions from probabilistic models as part of principled workflows for data analysis. Typical problems in Bayesian workflows are the approximation of intractable posterior distributions for diverse model types and the comparison of competing models of the same process in terms of the… ▽ More Modern Bayesian inference involves a mixture of computational techniques for estimating, validating, and drawing conclusions from probabilistic models as part of principled workflows for data analysis. Typical problems in Bayesian workflows are the approximation of intractable posterior distributions for diverse model types and the comparison of competing models of the same process in terms of their complexity and predictive performance. This manuscript introduces the Python library BayesFlow for simulation-based training of established neural network architectures for amortized data compression and inference. Amortized Bayesian inference, as implemented in BayesFlow, enables users to train custom neural networks on model simulations and re-use these networks for any subsequent application of the models. Since the trained networks can perform inference almost instantaneously, the upfront neural network training is quickly amortized. △ Less

Submitted 10 July, 2023; v1 submitted 28 June, 2023; originally announced June 2023.

Neural Superstatistics for Bayesian Estimation of Dynamic Cognitive Models

Authors: Lukas Schumacher, Paul-Christian Bürkner, Andreas Voss, Ullrich Köthe, Stefan T. Radev

Abstract: Mathematical models of cognition are often memoryless and ignore potential fluctuations of their parameters. However, human cognition is inherently dynamic. Thus, we propose to augment mechanistic cognitive models with a temporal dimension and estimate the resulting dynamics from a superstatistics perspective. Such a model entails a hierarchy between a low-level observation model and a high-level… ▽ More Mathematical models of cognition are often memoryless and ignore potential fluctuations of their parameters. However, human cognition is inherently dynamic. Thus, we propose to augment mechanistic cognitive models with a temporal dimension and estimate the resulting dynamics from a superstatistics perspective. Such a model entails a hierarchy between a low-level observation model and a high-level transition model. The observation model describes the local behavior of a system, and the transition model specifies how the parameters of the observation model evolve over time. To overcome the estimation challenges resulting from the complexity of superstatistical models, we develop and validate a simulation-based deep learning method for Bayesian inference, which can recover both time-varying and time-invariant parameters. We first benchmark our method against two existing frameworks capable of estimating time-varying parameters. We then apply our method to fit a dynamic version of the diffusion decision model to long time series of human response times data. Our results show that the deep learning approach is very efficient in capturing the temporal dynamics of the model. Furthermore, we show that the erroneous assumption of static or homogeneous parameters will hide important temporal information. △ Less

Submitted 20 September, 2023; v1 submitted 23 November, 2022; originally announced November 2022.

Censored autoregressive regression models with Student-$t$ innovations

Authors: Katherine A. L. Valeriano, Fernanda L. Schumacher, Christian E. Galarza, Larissa A. Matos

Abstract: The Student-$t$ distribution is widely used in statistical modeling of datasets involving outliers since its longer-than-normal tails provide a robust approach to hand such data. Furthermore, data collected over time may contain censored or missing observations, making it impossible to use standard statistical procedures. This paper proposes an algorithm to estimate the parameters of a censored li… ▽ More The Student-$t$ distribution is widely used in statistical modeling of datasets involving outliers since its longer-than-normal tails provide a robust approach to hand such data. Furthermore, data collected over time may contain censored or missing observations, making it impossible to use standard statistical procedures. This paper proposes an algorithm to estimate the parameters of a censored linear regression model when the regression errors are autocorrelated and the innovations follow a Student-$t$ distribution. To fit the proposed model, maximum likelihood estimates are obtained throughout the SAEM algorithm, which is a stochastic approximation of the EM algorithm useful for models in which the E-step does not have an analytic form. The methods are illustrated by the analysis of a real dataset that has left-censored and missing observations. We also conducted two simulations studies to examine the asymptotic properties of the estimates and the robustness of the model. △ Less

Submitted 19 January, 2022; v1 submitted 1 October, 2021; originally announced October 2021.

Comments: 22 pages, 10 figures and 3 tables

Canonical fundamental skew-t linear mixed models

Authors: Fernanda L. Schumacher, Larissa A. Matos, Celso R. B. Cabral

Abstract: In clinical trials, studies often present longitudinal data or clustered data. These studies are commonly analyzed using linear mixed models (LMMs), usually considering Gaussian assumptions for random effect and error terms. Recently, several proposals extended the restrictive assumptions from traditional LMM by more flexible ones that can accommodate skewness and heavy-tails and consequently are… ▽ More In clinical trials, studies often present longitudinal data or clustered data. These studies are commonly analyzed using linear mixed models (LMMs), usually considering Gaussian assumptions for random effect and error terms. Recently, several proposals extended the restrictive assumptions from traditional LMM by more flexible ones that can accommodate skewness and heavy-tails and consequently are more robust to outliers. This work proposes a canonical fundamental skew-t linear mixed model (ST-LMM), that allows for asymmetric and heavy-tailed random effects and errors and includes several important cases as special cases, which are presented and considered for model selection. For this robust and flexible model, we present an efficient EM-type algorithm for parameter estimation via maximum likelihood, implemented in a closed form by exploring the hierarchical representation of the ST-LMM. In addition, the estimation of standard errors and random effects is discussed. The methodology is illustrated through an application to schizophrenia data and some simulation studies. △ Less

Submitted 24 September, 2021; originally announced September 2021.

Approximate inferences for nonlinear mixed effects models with scale mixtures of skew-normal distributions

Authors: Fernanda L. Schumacher, Dipak K. Dey, Victor H. Lachos

Abstract: Nonlinear mixed effects models have received a great deal of attention in the statistical literature in recent years because of their flexibility in handling longitudinal studies, including human immunodeficiency virus viral dynamics, pharmacokinetic analyses, and studies of growth and decay. A standard assumption in nonlinear mixed effects models for continuous responses is that the random effect… ▽ More Nonlinear mixed effects models have received a great deal of attention in the statistical literature in recent years because of their flexibility in handling longitudinal studies, including human immunodeficiency virus viral dynamics, pharmacokinetic analyses, and studies of growth and decay. A standard assumption in nonlinear mixed effects models for continuous responses is that the random effects and the within-subject errors are normally distributed, making the model sensitive to outliers. We present a novel class of asymmetric nonlinear mixed effects models that provides efficient parameters estimation in the analysis of longitudinal data. We assume that, marginally, the random effects follow a multivariate scale mixtures of skew--normal distribution and that the random errors follow a symmetric scale mixtures of normal distribution, providing an appealing robust alternative to the usual normal distribution. We propose an approximate method for maximum likelihood estimation based on an EM-type algorithm that produces approximate maximum likelihood estimates and significantly reduces the numerical difficulties associated with the exact maximum likelihood estimation. Techniques for prediction of future responses under this class of distributions are also briefly discussed. The methodology is illustrated through an application to Theophylline kinetics data and through some simulating studies. △ Less

Submitted 29 July, 2020; originally announced July 2020.

A robust nonlinear mixed-effects model for COVID-19 deaths data

Authors: Fernanda L. Schumacher, Clecio S. Ferreira, Marcos O. Prates, Alberto Lachos, Victor H. Lachos

Abstract: The analysis of complex longitudinal data such as COVID-19 deaths is challenging due to several inherent features: (i) Similarly-shaped profiles with different decay patterns; (ii) Unexplained variation among repeated measurements within each country, these repeated measurements may be viewed as clustered data since they are taken on the same country at roughly the same time; (iii) Skewness, outli… ▽ More The analysis of complex longitudinal data such as COVID-19 deaths is challenging due to several inherent features: (i) Similarly-shaped profiles with different decay patterns; (ii) Unexplained variation among repeated measurements within each country, these repeated measurements may be viewed as clustered data since they are taken on the same country at roughly the same time; (iii) Skewness, outliers or skew-heavy-tailed noises are possibly embodied within response variables. This article formulates a robust nonlinear mixed-effects model based in the class of scale mixtures of skew-normal distributions for modeling COVID-19 deaths, which allows the analysts to model such data in the presence of the above described features simultaneously. An efficient EM-type algorithm is proposed to carry out maximum likelihood estimation of model parameters. The bootstrap method is used to determine inherent characteristics of the nonlinear individual profiles such as confidence interval of the predicted deaths and fitted curves. The target is to model COVID-19 deaths curves from some Latin American countries since this region is the new epicenter of the disease. Moreover, since a mixed-effect framework borrows information from the population-average effects, in our analysis we include some countries from Europe and North America that are in a more advanced stage of their COVID-19 deaths curve. △ Less

Submitted 1 August, 2020; v1 submitted 1 July, 2020; originally announced July 2020.

Comments: 11 pages, 2 figures, 4 tables

Scale mixture of skew-normal linear mixed models with within-subject serial dependence

Authors: Fernanda L. Schumacher, Victor H. Lachos, Larissa A. Matos

Abstract: In longitudinal studies, repeated measures are collected over time and hence they tend to be serially correlated. In this paper we consider an extension of skew-normal/independent linear mixed models introduced by Lachos et al. (2010), where the error term has a dependence structure, such as damped exponential correlation or autoregressive correlation of order p. The proposed model provides flexib… ▽ More In longitudinal studies, repeated measures are collected over time and hence they tend to be serially correlated. In this paper we consider an extension of skew-normal/independent linear mixed models introduced by Lachos et al. (2010), where the error term has a dependence structure, such as damped exponential correlation or autoregressive correlation of order p. The proposed model provides flexibility in capturing the effects of skewness and heavy tails simultaneously when continuous repeated measures are serially correlated. For this robust model, we present an efficient EM-type algorithm for computation of maximum likelihood estimation of parameters and the observed information matrix is derived analytically to account for standard errors. The methodology is illustrated through an application to schizophrenia data and several simulation studies. The proposed algorithm and methods are implemented in the new R package skewlmm. △ Less

Submitted 11 August, 2020; v1 submitted 3 February, 2020; originally announced February 2020.