Showing 1–12 of 12 results for author: Moustaki, I

Unfolding the Network of Peer Grades: A Latent Variable Approach

Authors: Giuseppe Mignemi, Yunxiao Chen, Irini Moustaki

Abstract: Peer grading is an educational system in which students assess each other's work. It is commonly applied under Massive Open Online Course (MOOC) and offline classroom settings. With this system, instructors receive a reduced grading workload, and students enhance their understanding of course materials by grading others' work. Peer grading data have a complex dependence structure, for which all th… ▽ More Peer grading is an educational system in which students assess each other's work. It is commonly applied under Massive Open Online Course (MOOC) and offline classroom settings. With this system, instructors receive a reduced grading workload, and students enhance their understanding of course materials by grading others' work. Peer grading data have a complex dependence structure, for which all the peer grades may be dependent. This complex dependence structure is due to a network structure of peer grading, where each student can be viewed as a vertex of the network, and each peer grade serves as an edge connecting one student as a grader to another student as an examinee. This paper introduces a latent variable model framework for analyzing peer grading data and develops a fully Bayesian procedure for its statistical inference. This framework has several advantages. First, when aggregating multiple peer grades, the average score and other simple summary statistics fail to account for grader effects and, thus, can be biased. The proposed approach produces more accurate model parameter estimates and, therefore, more accurate aggregated grades, by modeling the heterogeneous grading behavior with latent variables. Second, the proposed method provides a way to assess each student's performance as a grader, which may be used to identify a pool of reliable graders or generate feedback to help students improve their grading. Third, our model may further provide insights into the peer grading system by answering questions such as whether a student who performs better in coursework also tends to be a more reliable grader. Finally, thanks to the Bayesian approach, uncertainty quantification is straightforward when inferring the student-specific latent variables as well as the structural parameters of the model. The proposed method is applied to two real-world datasets. △ Less

Submitted 18 October, 2024; originally announced October 2024.

Learning High-dimensional Latent Variable Models via Doubly Stochastic Optimisation by Unadjusted Langevin

Authors: Motonori Oka, Yunxiao Chen, Irini Moustaki

Abstract: Latent variable models are widely used in social and behavioural sciences, such as education, psychology, and political science. In recent years, high-dimensional latent variable models have become increasingly common for analysing large and complex data. Estimating high-dimensional latent variable models using marginal maximum likelihood is computationally demanding due to the complexity of integ… ▽ More Latent variable models are widely used in social and behavioural sciences, such as education, psychology, and political science. In recent years, high-dimensional latent variable models have become increasingly common for analysing large and complex data. Estimating high-dimensional latent variable models using marginal maximum likelihood is computationally demanding due to the complexity of integrals involved. To address this challenge, stochastic optimisation, which combines stochastic approximation and sampling techniques, has been shown to be effective. This method iterates between two steps -- (1) sampling the latent variables from their posterior distribution based on the current parameter estimate, and (2) updating the fixed parameters using an approximate stochastic gradient constructed from the latent variable samples. In this paper, we propose a computationally more efficient stochastic optimisation algorithm. This improvement is achieved through the use of a minibatch of observations when sampling latent variables and constructing stochastic gradients, and an unadjusted Langevin sampler that utilises the gradient of the negative complete-data log-likelihood to sample latent variables. Theoretical results are established for the proposed algorithm, showing that the iterative parameter update converges to the marginal maximum likelihood estimate as the number of iterations goes to infinity. Furthermore, the proposed algorithm is shown to scale well to high-dimensional settings through simulation studies and a personality test application with 30,000 respondents, 300 items, and 30 latent dimensions. △ Less

Submitted 14 June, 2024; v1 submitted 13 June, 2024; originally announced June 2024.

The generalized Hausman test for detecting non-normality in the latent variable distribution of the two-parameter IRT model

Authors: Lucia Guastadisegni, Silvia Cagnone, Irini Moustaki, Vassilis Vasdekis

Abstract: This paper introduces the generalized Hausman test as a novel method for detecting non-normality of the latent variable distribution of unidimensional Item Response Theory (IRT) models for binary data. The test utilizes the pairwise maximum likelihood estimator obtained for the parameters of the classical two-parameter IRT model, which assumes normality of the latent variable, and the quasi-maximu… ▽ More This paper introduces the generalized Hausman test as a novel method for detecting non-normality of the latent variable distribution of unidimensional Item Response Theory (IRT) models for binary data. The test utilizes the pairwise maximum likelihood estimator obtained for the parameters of the classical two-parameter IRT model, which assumes normality of the latent variable, and the quasi-maximum likelihood estimator obtained under a semi-nonparametric framework, allowing for a more flexible distribution of the latent variable. The performance of the generalized Hausman test is evaluated through a simulation study and it is compared with the likelihood-ratio and the M2 test statistics. Additionally, various information criteria are computed. The simulation results show that the generalized Hausman test outperforms the other tests under most conditions. However, the results obtained from the information criteria are somewhat contradictory under certain conditions, suggesting a need for further investigation and interpretation. △ Less

Submitted 13 February, 2024; originally announced February 2024.

Pairwise likelihood estimation and limited information goodness-of-fit test statistics for binary factor analysis models under complex survey sampling

Authors: Haziq Jamil, Irini Moustaki, Chris Skinner

Abstract: This paper discusses estimation and limited information goodness-of-fit test statistics in factor models for binary data using pairwise likelihood estimation and sampling weights. The paper extends the applicability of pairwise likelihood estimation for factor models with binary data to accommodate complex sampling designs. Additionally, it introduces two key limited information test statistics: t… ▽ More This paper discusses estimation and limited information goodness-of-fit test statistics in factor models for binary data using pairwise likelihood estimation and sampling weights. The paper extends the applicability of pairwise likelihood estimation for factor models with binary data to accommodate complex sampling designs. Additionally, it introduces two key limited information test statistics: the Pearson chi-squared test and the Wald test. To enhance computational efficiency, the paper introduces modifications to both test statistics. The performance of the estimation and the proposed test statistics under simple random sampling and unequal probability sampling is evaluated using simulated data. △ Less

Submitted 23 July, 2024; v1 submitted 4 November, 2023; originally announced November 2023.

When Composite Likelihood Meets Stochastic Approximation

Authors: Giuseppe Alfonzetti, Ruggero Bellio, Yunxiao Chen, Irini Moustaki

Abstract: A composite likelihood is an inference function derived by multiplying a set of likelihood components. This approach provides a flexible framework for drawing inference when the likelihood function of a statistical model is computationally intractable. While composite likelihood has computational advantages, it can still be demanding when dealing with numerous likelihood components and a large sam… ▽ More A composite likelihood is an inference function derived by multiplying a set of likelihood components. This approach provides a flexible framework for drawing inference when the likelihood function of a statistical model is computationally intractable. While composite likelihood has computational advantages, it can still be demanding when dealing with numerous likelihood components and a large sample size. This paper tackles this challenge by employing an approximation of the conventional composite likelihood estimator, which is derived from an optimization procedure relying on stochastic gradients. This novel estimator is shown to be asymptotically normally distributed around the true parameter. In particular, based on the relative divergent rate of the sample size and the number of iterations of the optimization, the variance of the limiting distribution is shown to compound for two sources of uncertainty: the sampling variability of the data and the optimization noise, with the latter depending on the sampling distribution used to construct the stochastic gradients. The advantages of the proposed framework are illustrated through simulation studies on two working examples: an Ising model for binary data and a gamma frailty model for count data. Finally, a real-data application is presented, showing its effectiveness in a large-scale mental health survey. △ Less

Submitted 9 December, 2024; v1 submitted 6 October, 2023; originally announced October 2023.

DIF Analysis with Unknown Groups and Anchor Items

Authors: Gabriel Wallin, Yunxiao Chen, Irini Moustaki

Abstract: Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditi… ▽ More Ensuring fairness in instruments like survey questionnaires or educational tests is crucial. One way to address this is by a Differential Item Functioning (DIF) analysis, which examines if different subgroups respond differently to a particular item, controlling for their overall latent construct level. DIF analysis is typically conducted to assess measurement invariance at the item level. Traditional DIF analysis methods require knowing the comparison groups (reference and focal groups) and anchor items (a subset of DIF-free items). Such prior knowledge may not always be available, and psychometric methods have been proposed for DIF analysis when one piece of information is unknown. More specifically, when the comparison groups are unknown while anchor items are known, latent DIF analysis methods have been proposed that estimate the unknown groups by latent classes. When anchor items are unknown while comparison groups are known, methods have also been proposed, typically under a sparsity assumption -- the number of DIF items is not too large. However, DIF analysis when both pieces of information are unknown has not received much attention. This paper proposes a general statistical framework under this setting. In the proposed framework, we model the unknown groups by latent classes and introduce item-specific DIF parameters to capture the DIF effects. Assuming the number of DIF items is relatively small, an $L_1$-regularised estimator is proposed to simultaneously identify the latent classes and the DIF items. A computationally efficient Expectation-Maximisation (EM) algorithm is developed to solve the non-smooth optimisation problem for the regularised estimator. The performance of the proposed method is evaluated by simulation studies and an application to item response data from a real-world educational test. △ Less

Submitted 19 November, 2023; v1 submitted 1 May, 2023; originally announced May 2023.

Journal ref: Psychometrika 89 (2024) 267-295

Rotation to Sparse Loadings using $L^p$ Losses and Related Inference Problems

Authors: Xinyi Liu, Gabriel Wallin, Yunxiao Chen, Irini Moustaki

Abstract: Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper we propose a new family of oblique rotations based on component-wise $L^p$ loss functions $(0 < p\leq 1)$ that is closely related to… ▽ More Researchers have widely used exploratory factor analysis (EFA) to learn the latent structure underlying multivariate data. Rotation and regularised estimation are two classes of methods in EFA that they often use to find interpretable loading matrices. In this paper we propose a new family of oblique rotations based on component-wise $L^p$ loss functions $(0 < p\leq 1)$ that is closely related to an $L^p$ regularised estimator. We develop model selection and post-selection inference procedures based on the proposed rotation method. When the true loading matrix is sparse, the proposed method tends to outperform traditional rotation and regularised estimation methods in terms of statistical accuracy and computational cost. Since the proposed loss functions are nonsmooth, we develop an iteratively reweighted gradient projection algorithm for solving the optimisation problem. We also develop theoretical results that establish the statistical consistency of the estimation, model selection, and post-selection inference. We evaluate the proposed method and compare it with regularised estimation and traditional rotation methods via simulation studies. We further illustrate it using an application to the Big Five personality assessment. △ Less

Submitted 31 January, 2023; v1 submitted 5 June, 2022; originally announced June 2022.

Sequential Bayesian Inference for Factor Analysis

Authors: Konstantinos Vamvourellis, Konstantinos Kalogeropoulos, Irini Moustaki

Abstract: We develop an efficient Bayesian sequential inference framework for factor analysis models observed via various data types, such as continuous, binary and ordinal data. In the continuous data case, where it is possible to marginalise over the latent factors, the proposed methodology tailors the Iterated Batch Importance Sampling (IBIS) of Chopin (2002) to handle such models and we incorporate Hami… ▽ More We develop an efficient Bayesian sequential inference framework for factor analysis models observed via various data types, such as continuous, binary and ordinal data. In the continuous data case, where it is possible to marginalise over the latent factors, the proposed methodology tailors the Iterated Batch Importance Sampling (IBIS) of Chopin (2002) to handle such models and we incorporate Hamiltonian Markov Chain Monte Carlo. For binary and ordinal data, we develop an efficient IBIS scheme to handle the parameter and latent factors, combining with Laplace or Variational Bayes approximations. The methodology can be used in the context of sequential hypothesis testing via Bayes factors, which are known to have advantages over traditional null hypothesis testing. Moreover, the developed sequential framework offers multiple benefits even in non-sequential cases, by providing posterior distribution, model evidence and scoring rules (under the prequential framework) in one go, and by offering a more robust alternative computational scheme to Markov Chain Monte Carlo that can be useful in problematic target distributions. △ Less

Submitted 26 January, 2022; originally announced January 2022.

Model Assessment for a Generalised Bayesian Structural Equation Model

Authors: Konstantinos Vamvourellis, Konstantinos Kalogeropoulos, Irini Moustaki

Abstract: The paper proposes a novel model assessment paradigm aiming to address shortcoming of posterior predictive $p-$values, which provide the default metric of fit for Bayesian structural equation modelling (BSEM). The model framework of the paper focuses on the approximate zero approach, according to which parameters that would before set to zero (e.g. factor loadings) are now formulated to be approxi… ▽ More The paper proposes a novel model assessment paradigm aiming to address shortcoming of posterior predictive $p-$values, which provide the default metric of fit for Bayesian structural equation modelling (BSEM). The model framework of the paper focuses on the approximate zero approach, according to which parameters that would before set to zero (e.g. factor loadings) are now formulated to be approximate zero via informative priors (Muthen and Asparouhov, 2012). The introduced model assessment procedure monitors the out-of-sample predictive performance of the fitted model, and together with a list of guidelines we provide, one can investigate whether the hypothesised model is supported by the data. We incorporate scoring rules and cross-validation to supplement existing model assessment metrics for Bayesian SEM. The proposed tools can be applied to models for both categorical and continuous data. The modelling of categorical and non-normally distributed continuous data is facilitated with the introduction of an item-individual random effect that can also be used for outlier detection. We study the performance of the proposed methodology via simulations. The factor model for continuous and binary data is fitted to data on the `Big-5' personality scale and the Fagerstrom test for nicotine dependence respectively. △ Less

Submitted 28 June, 2022; v1 submitted 4 April, 2021; originally announced April 2021.

A Note on Likelihood Ratio Tests for Models with Latent Variables

Authors: Yunxiao Chen, Irini Moustaki, Haoran Zhang

Abstract: The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks' theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a $χ^2$-distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison.… ▽ More The likelihood ratio test (LRT) is widely used for comparing the relative fit of nested latent variable models. Following Wilks' theorem, the LRT is conducted by comparing the LRT statistic with its asymptotic distribution under the restricted model, a $χ^2$-distribution with degrees of freedom equal to the difference in the number of free parameters between the two nested models under comparison. For models with latent variables such as factor analysis, structural equation models and random effects models, however, it is often found that the $χ^2$ approximation does not hold. In this note, we show how the regularity conditions of Wilks' theorem may be violated using three examples of models with latent variables. In addition, a more general theory for LRT is given that provides the correct asymptotic theory for these LRTs. This general theory was first established in Chernoff (1954) and discussed in both van der Vaart (2000) and Drton (2009), but it does not seem to have received enough attention. We illustrate this general theory with the three examples. △ Less

Submitted 21 November, 2020; v1 submitted 10 August, 2020; originally announced August 2020.

Journal ref: Psychometrika 85 (2020) 996-1012

Detection of Two-Way Outliers in Multivariate Data and Application to Cheating Detection in Educational Tests

Authors: Yunxiao Chen, Yan Lu, Irini Moustaki

Abstract: The paper proposes a new latent variable model for the simultaneous (two-way) detection of outlying individuals and items for item-response-type data. The proposed model is a synergy between a factor model for binary responses and continuous response times that captures normal item response behaviour and a latent class model that captures the outlying individuals and items. A statistical decision… ▽ More The paper proposes a new latent variable model for the simultaneous (two-way) detection of outlying individuals and items for item-response-type data. The proposed model is a synergy between a factor model for binary responses and continuous response times that captures normal item response behaviour and a latent class model that captures the outlying individuals and items. A statistical decision framework is developed under the proposed model that provides compound decision rules for controlling local false discovery/nondiscovery rates of outlier detection. Statistical inference is carried out under a Bayesian framework, for which a Markov chain Monte Carlo algorithm is developed. The proposed method is applied to the detection of cheating in educational tests due to item leakage using a case study of a computer-based nonadaptive licensure assessment. The performance of the proposed method is evaluated by simulation studies. △ Less

Submitted 22 October, 2021; v1 submitted 21 November, 2019; originally announced November 2019.

Explaining the behavior of joint and marginal Monte Carlo estimators in latent variable models with independence assumptions

Authors: Silia Vitoratou, Ioannis Ntzoufras, Irini Moustaki

Abstract: In latent variable models the parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Baye… ▽ More In latent variable models the parameter estimation can be implemented by using the joint or the marginal likelihood, based on independence or conditional independence assumptions. The same dilemma occurs within the Bayesian framework with respect to the estimation of the Bayesian marginal (or integrated) likelihood, which is the main tool for model comparison and averaging. In most cases, the Bayesian marginal likelihood is a high dimensional integral that cannot be computed analytically and a plethora of methods based on Monte Carlo integration (MCI) are used for its estimation. In this work, it is shown that the joint MCI approach makes subtle use of the properties of the adopted model, leading to increased error and bias in finite settings. The sources and the components of the error associated with estimators under the two approaches are identified here and provided in exact forms. Additionally, the effect of the sample covariation on the Monte Carlo estimators is examined. In particular, even under independence assumptions the sample covariance will be close to (but not exactly) zero which surprisingly has a severe effect on the estimated values and their variability. To address this problem, an index of the sample's divergence from independence is introduced as a multivariate extension of covariance. The implications addressed here are important in the majority of practical problems appearing in Bayesian inference of multi-parameter models with analogous structures. △ Less

Submitted 4 November, 2013; originally announced November 2013.