-
Testing and Improving the Robustness of Amortized Bayesian Inference for Cognitive Models
Authors:
Yufei Wu,
Stefan Radev,
Francis Tuerlinckx
Abstract:
Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models, which are statistical models representing cognitive processes. In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks. To this end, we conduct systematic analyses on a toy example and analyze both synthetic a…
▽ More
Contaminant observations and outliers often cause problems when estimating the parameters of cognitive models, which are statistical models representing cognitive processes. In this study, we test and improve the robustness of parameter estimation using amortized Bayesian inference (ABI) with neural networks. To this end, we conduct systematic analyses on a toy example and analyze both synthetic and real data using a popular cognitive model, the Drift Diffusion Models (DDM). First, we study the sensitivity of ABI to contaminants with tools from robust statistics: the empirical influence function and the breakdown point. Next, we propose a data augmentation or noise injection approach that incorporates a contamination distribution into the data-generating process during training. We examine several candidate distributions and evaluate their performance and cost in terms of accuracy and efficiency loss relative to a standard estimator. Introducing contaminants from a Cauchy distribution during training considerably increases the robustness of the neural density estimator as measured by bounded influence functions and a much higher breakdown point. Overall, the proposed method is straightforward and practical to implement and has a broad applicability in fields where outlier detection or removal is challenging.
△ Less
Submitted 29 December, 2024;
originally announced December 2024.
-
A geometrical perspective on parametric psychometric models
Authors:
Francis Tuerlinckx
Abstract:
Psychometrics and quantitative psychology rely strongly on statistical models to measure psychological processes. As a branch of mathematics, geometry is inherently connected to measurement and focuses on properties such as distance and volume. However, despite the common root of measurement, geometry is currently not used a lot in psychological measurement. In this paper, my aim is to illustrate…
▽ More
Psychometrics and quantitative psychology rely strongly on statistical models to measure psychological processes. As a branch of mathematics, geometry is inherently connected to measurement and focuses on properties such as distance and volume. However, despite the common root of measurement, geometry is currently not used a lot in psychological measurement. In this paper, my aim is to illustrate how ideas from non-Euclidean geometry may be relevant for psychometrics.
△ Less
Submitted 16 October, 2024;
originally announced October 2024.
-
Prepaid parameter estimation without likelihoods
Authors:
Merijn Mestdagh,
Stijn Verdonck,
Kristof Meers,
Tim Loossens,
Francis Tuerlinckx
Abstract:
In various fields, statistical models of interest are analytically intractable. As a result, statistical inference is greatly hampered by computational constraints. However, given a model, different users with different data are likely to perform similar computations. Computations done by one user are potentially useful for other users with different data sets. We propose a pooling of resources ac…
▽ More
In various fields, statistical models of interest are analytically intractable. As a result, statistical inference is greatly hampered by computational constraints. However, given a model, different users with different data are likely to perform similar computations. Computations done by one user are potentially useful for other users with different data sets. We propose a pooling of resources across researchers to capitalize on this. More specifically, we preemptively chart out the entire space of possible model outcomes in a prepaid database. Using advanced interpolation techniques, any individual estimation problem can now be solved on the spot. The prepaid method can easily accommodate different priors as well as constraints on the parameters. We created prepaid databases for three challenging models and demonstrate how they can be distributed through an online parameter estimation service. Our method outperforms state-of-the-art estimation techniques in both speed (with a 23,000 to 100,000-fold speed up) and accuracy, and is able to handle previously quasi inestimable models.
△ Less
Submitted 23 December, 2018;
originally announced December 2018.
