A General Framework for Design-Based Treatment Effect Estimation in Paired Cluster-Randomized Experiments
Authors:
Charlotte Z. Mann,
Adam C. Sales,
Johann A. Gagnon-Bartsch
Abstract:
Paired cluster-randomized experiments (pCRTs) are common across many disciplines because there is often natural clustering of individuals, and paired randomization can help balance baseline covariates to improve experimental precision. Although pCRTs are common, there is surprisingly no obvious way to analyze this randomization design if an individual-level (rather than cluster-level) treatment ef…
▽ More
Paired cluster-randomized experiments (pCRTs) are common across many disciplines because there is often natural clustering of individuals, and paired randomization can help balance baseline covariates to improve experimental precision. Although pCRTs are common, there is surprisingly no obvious way to analyze this randomization design if an individual-level (rather than cluster-level) treatment effect is of interest. Variance estimation is also complicated due to the dependency created through pairing clusters. Therefore, we aim to provide an intuitive and practical comparison between different estimation strategies in pCRTs in order to inform practitioners' choice of strategy. To this end, we present a general framework for design-based estimation in pCRTs for average individual effects. This framework offers a novel and intuitive view on the bias-variance trade-off between estimators and emphasizes the benefits of covariate adjustment for estimation with pCRTs. In addition to providing a general framework for estimation in pCRTs, the point and variance estimators we present support fixed-sample unbiased estimation with similar precision to a common regression model and consistently conservative variance estimation. Through simulation studies, we compare the performance of the point and variance estimators reviewed. Finally, we compare the performance of estimators with simulations using real data from an educational efficacy trial. Our analysis and simulation studies inform the choice of point and variance estimators for analyzing pCRTs in practice.
△ Less
Submitted 1 July, 2024;
originally announced July 2024.
Combining observational and experimental data for causal inference considering data privacy
Authors:
Charlotte Z. Mann,
Adam C. Sales,
Johann A. Gagnon-Bartsch
Abstract:
Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive information might be tolerable to organizations that hou…
▽ More
Combining observational and experimental data for causal inference can improve treatment effect estimation. However, many observational data sets cannot be released due to data privacy considerations, so one researcher may not have access to both experimental and observational data. Nonetheless, a small amount of risk of disclosing sensitive information might be tolerable to organizations that house confidential data. In these cases, organizations can employ data privacy techniques, which decrease disclosure risk, potentially at the expense of data utility. In this paper, we explore disclosure limiting transformations of observational data, which can be combined with experimental data to estimate the sample and population average treatment effects. We consider leveraging observational data to improve generalizability of treatment effect estimates when a randomized experiment (RCT) is not representative of the population of interest, and to increase precision of treatment effect estimates. Through simulation studies, we illustrate the trade-off between privacy and utility when employing different disclosure limiting transformations. We find that leveraging transformed observational data in treatment effect estimation can still improve estimation over only using data from an RCT.
△ Less
Submitted 23 August, 2024; v1 submitted 5 August, 2023;
originally announced August 2023.
