Assessing the Impact of Covariate Distribution and Positivity Violation on Weighting-Based Indirect Comparisons: a Simulation Study
Authors:
Arnaud Serret-Larmande,
Jérôme Lambert,
Stéphane Gaudry,
David Hajage
Abstract:
Population-Adjusted Indirect Comparisons (PAICs) are used to estimate treatment effects when direct comparisons are infeasible and individual patient data (IPD) are only available for one trial. Among PAIC methods, Matching-Adjusted Indirect Comparison (MAIC) is the most widely used. However, little is known about how MAIC performs under challenging conditions such as limited covariate overlap or…
▽ More
Population-Adjusted Indirect Comparisons (PAICs) are used to estimate treatment effects when direct comparisons are infeasible and individual patient data (IPD) are only available for one trial. Among PAIC methods, Matching-Adjusted Indirect Comparison (MAIC) is the most widely used. However, little is known about how MAIC performs under challenging conditions such as limited covariate overlap or markedly non-normal covariate distributions.
We conducted a Monte Carlo simulation study comparing three estimators: (i) MAIC matching first moment (MAIC-1), (ii) MAIC matching first and second moments (MAIC-2), and (iii) a benchmark method leveraging full IPD -- Propensity Score Weighting (PSW). We examined eight scenarios ranging from ideal conditions to situations with positivity violations and non-normal (including bimodal) covariate distributions. We assessed both anchored and unanchored estimators and examined the impact of adjustment model misspecification. We also applied these estimators to real-world data from the AKIKI and AKIKI-2 trials, comparing renal replacement therapy strategies in critically ill patients.
MAIC-1 demonstrated robust performance, remaining unbiased in the presence of moderate positivity violations and non-normal covariates, while MAIC-2 and PSW appeared more sensitive to positivity violations. All methods showed substantial bias when key confounders were omitted, emphasizing the importance of correct model specification. In real-world data, a consistent trend was found with MAIC-1 showing narrower confidence intervals with positivity violation.
Our findings support the cautious use of unanchored MAICs and highlight MAIC-1's resilience across moderate violations of assumptions. However, the method's limited flexibility underscores the need for careful use in real-world settings.
△ Less
Submitted 20 July, 2025; v1 submitted 16 July, 2025;
originally announced July 2025.
Closed-form variance estimators for weighted and stratified dose-response function estimators using generalized propensity score
Authors:
Valérie Garès,
Guillaume Chauvet,
David Hajage
Abstract:
Propensity score methods are widely used in observational studies for evaluating marginal treatment effects. The generalized propensity score (GPS) is an extension of the propensity score framework, historically developed in the case of binary exposures, for use with quantitative or continuous exposures. In this paper, we proposed variance esti-mators for treatment effect estimators on continuous…
▽ More
Propensity score methods are widely used in observational studies for evaluating marginal treatment effects. The generalized propensity score (GPS) is an extension of the propensity score framework, historically developed in the case of binary exposures, for use with quantitative or continuous exposures. In this paper, we proposed variance esti-mators for treatment effect estimators on continuous outcomes. Dose-response functions (DRF) were estimated through weighting on the inverse of the GPS, or using stratification. Variance estimators were evaluated using Monte Carlo simulations. Despite the use of stabilized weights, the variability of the weighted estimator of the DRF was particularly high, and none of the variance estimators (a bootstrap-based estimator, a closed-form estimator especially developped to take into account the estimation step of the GPS, and a sandwich estimator) were able to adequately capture this variability, resulting in coverages below to the nominal value, particularly when the proportion of the variation in the quantitative exposure explained by the covariates was 1 large. The stratified estimator was more stable, and variance estima-tors (a bootstrap-based estimator, a pooled linearized estimator, and a pooled model-based estimator) more efficient at capturing the empirical variability of the parameters of the DRF. The pooled variance estimators tended to overestimate the variance, whereas the bootstrap estimator, which intrinsically takes into account the estimation step of the GPS, resulted in correct variance estimations and coverage rates. These methods were applied to a real data set with the aim of assessing the effect of maternal body mass index on newborn birth weight.
△ Less
Submitted 6 July, 2020;
originally announced July 2020.
