Mixed Effects Models are Sometimes Terrible
Authors:
Christopher Eager,
Joseph Roy
Abstract:
Mixed-effects models have emerged as the gold standard of statistical analysis in different sub-fields of linguistics (Baayen, Davidson & Bates, 2008; Johnson, 2009; Barr, et al, 2013; Gries, 2015). One problematic feature of these models is their failure to converge under maximal (or even near-maximal) random effects structures. The lack of convergence is relatively unaddressed in linguistics and…
▽ More
Mixed-effects models have emerged as the gold standard of statistical analysis in different sub-fields of linguistics (Baayen, Davidson & Bates, 2008; Johnson, 2009; Barr, et al, 2013; Gries, 2015). One problematic feature of these models is their failure to converge under maximal (or even near-maximal) random effects structures. The lack of convergence is relatively unaddressed in linguistics and when it is addressed has resulted in statistical practices (e.g. Jaeger, 2009; Gries, 2015; Bates, et al, 2015b) that are premised on the idea that non-convergence is an indication that a random effects structure is over-specified (or not parsimonious), the parsimonious convergence hypothesis (PCH). We test the PCH by running simulations in lme4 under two sets of assumptions for both a linear dependent variable and a binary dependent variable in order to assess the rate of non-convergence for both types of mixed effects models when a known maximal effect structure is used to generate the data (i.e. when non-convergence cannot be explained by random effects with zero variance). Under the PCH, lack of convergence is treated as evidence against a more maximal random effects structure, but that result is not upheld with our simulations. We provide an alternative model, fully specified Bayesian models implemented in rstan (Stan Development Team, 2016; Carpenter, et al, in press) that removed the convergence problems almost entirely in simulations of the same conditions. These results indicate that when there is known non-zero variance for all slopes and intercepts, under realistic distributions of data and with moderate to severe imbalance, mixed effects models in lme4 have moderate to high non-convergence rates which can cause linguistic researchers to wrongfully exclude random effect terms.
△ Less
Submitted 5 January, 2017;
originally announced January 2017.
Confronting Quasi-Separation in Logistic Mixed Effects for Linguistic Data: A Bayesian Approach
Authors:
Amelia Kimball,
Kailen Shantz,
Christopher Eager,
Joseph Roy
Abstract:
Mixed effects regression models are widely used by language researchers. However, these regressions are implemented with an algorithm which may not converge on a solution. While convergence issues in linear mixed effects models can often be addressed with careful experiment design and model building, logistic mixed effects models introduce the possibility of separation or quasi-separation, which c…
▽ More
Mixed effects regression models are widely used by language researchers. However, these regressions are implemented with an algorithm which may not converge on a solution. While convergence issues in linear mixed effects models can often be addressed with careful experiment design and model building, logistic mixed effects models introduce the possibility of separation or quasi-separation, which can cause problems for model estimation that result in convergence errors or in unreasonable model estimates. These problems cannot be solved by experiment or model design. In this paper, we discuss (quasi-)separation with the language researcher in mind, explaining what it is, how it causes problems for model estimation, and why it can be expected in linguistic datasets. Using real linguistic datasets, we then show how Bayesian models can be used to overcome convergence issues introduced by quasi-separation, whereas frequentist approaches fail. On the basis of these demonstrations, we advocate for the adoption of Bayesian models as a practical solution to dealing with convergence issues when modeling binary linguistic data.
△ Less
Submitted 7 September, 2018; v1 submitted 31 October, 2016;
originally announced November 2016.
