-
Optimal treatment regimes for the net benefit of a treatment
Authors:
François Petit,
Gérard Biau,
Raphaël Porcher
Abstract:
We develop a mathematical framework to define an optimal individualized treatment rule (ITR) within the context of prioritized outcomes in a randomized controlled trial. Our optimality criterion is based on the framework of generalized pairwise comparisons. We propose two approaches for estimating optimal ITRs on a pairwise basis. The first approach is a variant of the k-nearest neighbors algorith…
▽ More
We develop a mathematical framework to define an optimal individualized treatment rule (ITR) within the context of prioritized outcomes in a randomized controlled trial. Our optimality criterion is based on the framework of generalized pairwise comparisons. We propose two approaches for estimating optimal ITRs on a pairwise basis. The first approach is a variant of the k-nearest neighbors algorithm. The second approach is a meta-learning method based on a randomized bagging scheme, which enables the use of any classification algorithm to construct an ITR. We investigate the theoretical properties of these estimation procedures, evaluate their performance through Monte Carlo simulations, and demonstrate their application to clinical trial data.
△ Less
Submitted 16 June, 2025; v1 submitted 28 March, 2025;
originally announced March 2025.
-
Benchmarking covariates balancing methods, a simulation study
Authors:
Etienne Peyrot,
Raphaël Porcher,
Francois Petit
Abstract:
Causal inference in observational studies has advanced significantly since Rosenbaum and Rubin introduced propensity score methods. Inverse probability of treatment weighting (IPTW) is widely used to handle confounding bias. However, newer methods, such as energy balancing (EB), kernel optimal matching (KOM), and covariate balancing propensity score by tailored loss function (TLF), offer model-fre…
▽ More
Causal inference in observational studies has advanced significantly since Rosenbaum and Rubin introduced propensity score methods. Inverse probability of treatment weighting (IPTW) is widely used to handle confounding bias. However, newer methods, such as energy balancing (EB), kernel optimal matching (KOM), and covariate balancing propensity score by tailored loss function (TLF), offer model-free or non-parametric alternatives. Despite these developments, guidance remains limited in selecting the most suitable method for treatment effect estimation in practical applications. This study compares IPTW with EB, KOM, and TLF, focusing on their ability to estimate treatment effects since this is the primary objective in many applications. Monte Carlo simulations are used to assess the ability of these balancing methods combined with different estimators to evaluate average treatment effect. We compare these methods across a range of scenarios varying sample size, level of confusion, and proportion of treated. In our simulation, we observe no significant advantages in using EB, KOM, or TLF methods over IPTW. Moreover, these recent methods make obtaining confidence intervals with nominal coverage difficult. We also compare the methods on the PROBITsim dataset and get results similar to those of our simulations.
△ Less
Submitted 29 November, 2024;
originally announced December 2024.
-
G-computation for increasing performances of clinical trials with individual randomization and binary response
Authors:
Joe de Keizer,
Rémi Lenain,
Raphaël Porcher,
Sarah Zoha,
Arthur Chatton,
Yohann Foucher
Abstract:
In a clinical trial, the random allocation aims to balance prognostic factors between arms, preventing true confounders. However, residual differences due to chance may introduce near-confounders. Adjusting on prognostic factors is therefore recommended, especially because the related increase of the power. In this paper, we hypothesized that G-computation associated with machine learning could be…
▽ More
In a clinical trial, the random allocation aims to balance prognostic factors between arms, preventing true confounders. However, residual differences due to chance may introduce near-confounders. Adjusting on prognostic factors is therefore recommended, especially because the related increase of the power. In this paper, we hypothesized that G-computation associated with machine learning could be a suitable method for randomized clinical trials even with small sample sizes. It allows for flexible estimation of the outcome model, even when the covariates' relationships with outcomes are complex. Through simulations, penalized regressions (Lasso, Elasticnet) and algorithm-based methods (neural network, support vector machine, super learner) were compared. Penalized regressions reduced variance but may introduce a slight increase in bias. The associated reductions in sample size ranged from 17\% to 54\%. In contrast, algorithm-based methods, while effective for larger and more complex data structures, underestimated the standard deviation, especially with small sample sizes. In conclusion, G-computation with penalized models, particularly Elasticnet with splines when appropriate, represents a relevant approach for increasing the power of RCTs and accounting for potential near-confounders.
△ Less
Submitted 15 November, 2024;
originally announced November 2024.
-
Clinical research and methodology What usage and what hierarchical order for secondary endpoints?
Authors:
Silvy Laporte,
Marine Diviné,
Danièle Girault,
Pierre Boutouyrie,
Olivier Chassany,
Michel Cucherat,
Hervé de Trogoff,
Sophie Dubois,
Cecile Fouret,
Natalie Hoog-Labouret,
Pascale Jolliet,
Patrick Mismetti,
Raphaël Porcher,
Cécile Rey-Coquais,
Eric Vicaut
Abstract:
In a randomised clinical trial, when the result of the primary endpoint shows a significant benefit, the secondary endpoints are scrutinised to identify additional effects of the treatment. However, this approach entails a risk of concluding that there is a benefit for one of these endpoints when such benefit does not exist (inflation of type I error risk). There are mainly two methods used to con…
▽ More
In a randomised clinical trial, when the result of the primary endpoint shows a significant benefit, the secondary endpoints are scrutinised to identify additional effects of the treatment. However, this approach entails a risk of concluding that there is a benefit for one of these endpoints when such benefit does not exist (inflation of type I error risk). There are mainly two methods used to control the risk of drawing erroneous conclusions for secondary endpoints. The first method consists of distributing the risk over several co-primary endpoints, so as to maintain an overall risk of 5%. The second is the hierarchical test procedure, which consists of first establishing a hierarchy of the endpoints, then evaluating each endpoint in succession according to this hierarchy while the endpoints continue to show statistical significance. This simple method makes it possible to show the additional advantages of treatments and to identify the factors that differentiate them.
△ Less
Submitted 23 September, 2024;
originally announced September 2024.
-
Estimating Complier Average Causal Effects with Mixtures of Experts
Authors:
François Grolleau,
Céline Béji,
Raphaël Porcher,
François Petit
Abstract:
Treatment non-compliance, where individuals deviate from their assigned experimental conditions, frequently complicates the estimation of causal effects. To address this, we introduce a novel learning framework based on a mixture of experts architecture to estimate the Complier Average Causal Effect (CACE). Our framework provides a flexible alternative to classical instrumental variable methods by…
▽ More
Treatment non-compliance, where individuals deviate from their assigned experimental conditions, frequently complicates the estimation of causal effects. To address this, we introduce a novel learning framework based on a mixture of experts architecture to estimate the Complier Average Causal Effect (CACE). Our framework provides a flexible alternative to classical instrumental variable methods by relaxing their strict monotonicity and exclusion restriction assumptions. We develop a principled, two-step procedure where each step is optimized with a dedicated Expectation-Maximization (EM) algorithm. Crucially, we provide formal proofs that the model's components are identifiable, ensuring the learning procedure is well-posed. The resulting CACE estimators are proven to be consistent and asymptotically normal. Extensive simulations demonstrate that our method achieves a substantially lower root mean squared error than traditional instrumental variable approaches when their assumptions fail, an advantage that persists even when our own mixture of experts are misspecified. We illustrate the framework's practical utility on data from a large-scale randomized trial.
△ Less
Submitted 24 June, 2025; v1 submitted 4 May, 2024;
originally announced May 2024.
-
Do machine learning methods lead to similar individualized treatment rules? A comparison study on real data
Authors:
Florie Bouvier,
Etienne Peyrot,
Alan Balendran,
Corentin Ségalas,
Ian Roberts,
François Petit,
Raphaël Porcher
Abstract:
Identifying patients who benefit from a treatment is a key aspect of personalized medicine, which allows the development of individualized treatment rules (ITRs). Many machine learning methods have been proposed to create such rules. However, to what extent the methods lead to similar ITRs, i.e., recommending the same treatment for the same individuals is unclear. In this work, we compared 22 of t…
▽ More
Identifying patients who benefit from a treatment is a key aspect of personalized medicine, which allows the development of individualized treatment rules (ITRs). Many machine learning methods have been proposed to create such rules. However, to what extent the methods lead to similar ITRs, i.e., recommending the same treatment for the same individuals is unclear. In this work, we compared 22 of the most common approaches in two randomized control trials. Two classes of methods can be distinguished. The first class of methods relies on predicting individualized treatment effects from which an ITR is derived by recommending the treatment evaluated to the individuals with a predicted benefit. In the second class, methods directly estimate the ITR without estimating individualized treatment effects. For each trial, the performance of ITRs was assessed by various metrics, and the pairwise agreement between all ITRs was also calculated. Results showed that the ITRs obtained via the different methods generally had considerable disagreements regarding the patients to be treated. A better concordance was found among akin methods. Overall, when evaluating the performance of ITRs in a validation sample, all methods produced ITRs with limited performance, suggesting a high potential for optimism. For non-parametric methods, this optimism was likely due to overfitting. The different methods do not lead to similar ITRs and are therefore not interchangeable. The choice of the method strongly influences for which patients a certain treatment is recommended, drawing some concerns about their practical use.
△ Less
Submitted 30 January, 2024; v1 submitted 7 August, 2023;
originally announced August 2023.
-
A Comprehensive Framework for the Evaluation of Individual Treatment Rules From Observational Data
Authors:
François Grolleau,
Francois Petit,
Raphaël Porcher
Abstract:
Individualized treatment rules (ITRs) are deterministic decision rules that recommend treatments to individuals based on their characteristics. Though ubiquitous in medicine, ITRs are hardly ever evaluated in randomized controlled trials. To evaluate ITRs from observational data, we introduce a new probabilistic model and distinguish two situations: i) the situation of a newly developed ITR, where…
▽ More
Individualized treatment rules (ITRs) are deterministic decision rules that recommend treatments to individuals based on their characteristics. Though ubiquitous in medicine, ITRs are hardly ever evaluated in randomized controlled trials. To evaluate ITRs from observational data, we introduce a new probabilistic model and distinguish two situations: i) the situation of a newly developed ITR, where data are from a population where no patient implements the ITR, and ii) the situation of a partially implemented ITR, where data are from a population where the ITR is implemented in some unidentified patients. In the former situation, we propose a procedure to explore the impact of an ITR under various implementation schemes. In the latter situation, on top of the fundamental problem of causal inference, we need to handle an additional latent variable denoting implementation. To evaluate ITRs in this situation, we propose an estimation procedure that relies on an expectation-maximization algorithm. In Monte Carlo simulations our estimators appear unbiased with confidence intervals achieving nominal coverage. We illustrate our approach on the MIMIC-III database, focusing on ITRs for dialysis initiation in patients with acute kidney injury.
△ Less
Submitted 21 August, 2023; v1 submitted 13 July, 2022;
originally announced July 2022.
-
How to improve the quality of comparisons using external control cohorts in single-arm clinical trials?
Authors:
Jérôme Lambert,
Etienne Lengline,
Raphaël Porcher,
Rodolphe Thiébaut,
Sarah Zohar,
Sylvie Chevret
Abstract:
PURPOSE Providing rapid answers and early acces to patients to innovative treatments without randomized clinical trial (RCT) is growing, with benefit estimated from single-arm trials. This has become common in oncology, impacting the approval pathway of health technology assessment agencies. We aimed to provide some guidance for indirect comparison to external controls to improve the level of evid…
▽ More
PURPOSE Providing rapid answers and early acces to patients to innovative treatments without randomized clinical trial (RCT) is growing, with benefit estimated from single-arm trials. This has become common in oncology, impacting the approval pathway of health technology assessment agencies. We aimed to provide some guidance for indirect comparison to external controls to improve the level of evidence following such uncontrolled designs. METHODS We used the illustrative example of blinatumomab, a bispecific antibody for the treatment of B-cell ALL in complete remission (CR) with persistent minimal residual disease (MRD). Its approval relied on a single-arm trial conducted in 86 adults with B-cell ALL in CR, with undetectable MRD after one cycle as the main endpoint. To maximize the validity of indirect comparisons, a 3-step process for incorporating external control data to such single-arm trial data is proposed and detailed, with emphasis on the example. RESULTS The first step includes the definition of estimand, i.e. the treatment effect reflecting the clinical question. The second step relies on the adequate selection of external controls, from previous RCT or real-world data (RWD) obtained from patient cohort, registries, or electronic patient files. The third step consists in chosing the statistical approach targeting the treatment effect of interest, either in the whole population or restricted to the single-arm trial or the external controls, and depending on the available individual-level or aggregrated external data. CONCLUSION Validity of treatment effect derived from indirect comparisons heavily depends on carefull methodological considerations that are included in the proposed 3-step procedure. Because the level of evidence of a well conducted RCT cannot be guaranteed, post-market authorization evaluation is even more important than in standard settings.
△ Less
Submitted 20 June, 2022;
originally announced June 2022.
-
Epidemic Models for COVID-19 during the First Wave from February to May 2020: a Methodological Review
Authors:
Marie Garin,
Myrto Limnios,
Alice Nicolaï,
Ioannis Bargiotas,
Olivier Boulant,
Stephen Chick,
Amir Dib,
Theodoros Evgeniou,
Mathilde Fekom,
Argyris Kalogeratos,
Christophe Labourdette,
Anton Ovchinnikov,
Raphaël Porcher,
Camille Pouchol,
Nicolas Vayatis
Abstract:
We review epidemiological models for the propagation of the COVID-19 pandemic during the early months of the outbreak: from February to May 2020. The aim is to propose a methodological review that highlights the following characteristics: (i) the epidemic propagation models, (ii) the modeling of intervention strategies, (iii) the models and estimation procedures of the epidemic parameters and (iv)…
▽ More
We review epidemiological models for the propagation of the COVID-19 pandemic during the early months of the outbreak: from February to May 2020. The aim is to propose a methodological review that highlights the following characteristics: (i) the epidemic propagation models, (ii) the modeling of intervention strategies, (iii) the models and estimation procedures of the epidemic parameters and (iv) the characteristics of the data used. We finally selected 80 articles from open access databases based on criteria such as the theoretical background, the reproducibility, the incorporation of interventions strategies, etc. It mainly resulted to phenomenological, compartmental and individual-level models. A digital companion including an online sheet, a Kibana interface and a markdown document is proposed. Finally, this work provides an opportunity to witness how the scientific community reacted to this unique situation.
△ Less
Submitted 3 September, 2021;
originally announced September 2021.
-
Wasserstein Random Forests and Applications in Heterogeneous Treatment Effects
Authors:
Qiming Du,
Gérard Biau,
François Petit,
Raphaël Porcher
Abstract:
We present new insights into causal inference in the context of Heterogeneous Treatment Effects by proposing natural variants of Random Forests to estimate the key conditional distributions. To achieve this, we recast Breiman's original splitting criterion in terms of Wasserstein distances between empirical measures. This reformulation indicates that Random Forests are well adapted to estimate con…
▽ More
We present new insights into causal inference in the context of Heterogeneous Treatment Effects by proposing natural variants of Random Forests to estimate the key conditional distributions. To achieve this, we recast Breiman's original splitting criterion in terms of Wasserstein distances between empirical measures. This reformulation indicates that Random Forests are well adapted to estimate conditional distributions and provides a natural extension of the algorithm to multivariate outputs. Following the philosophy of Breiman's construction, we propose some variants of the splitting rule that are well-suited to the conditional distribution estimation problem. Some preliminary theoretical connections are established along with various numerical experiments, which show how our approach may help to conduct more transparent causal inference in complex situations.
△ Less
Submitted 15 February, 2021; v1 submitted 8 June, 2020;
originally announced June 2020.
-
Stratified cross-validation for unbiased and privacy-preserving federated learning
Authors:
R. Bey,
R. Goussault,
M. Benchoufi,
R. Porcher
Abstract:
Large-scale collections of electronic records constitute both an opportunity for the development of more accurate prediction models and a threat for privacy. To limit privacy exposure new privacy-enhancing techniques are emerging such as federated learning which enables large-scale data analysis while avoiding the centralization of records in a unique database that would represent a critical point…
▽ More
Large-scale collections of electronic records constitute both an opportunity for the development of more accurate prediction models and a threat for privacy. To limit privacy exposure new privacy-enhancing techniques are emerging such as federated learning which enables large-scale data analysis while avoiding the centralization of records in a unique database that would represent a critical point of failure. Although promising regarding privacy protection, federated learning prevents using some data-cleaning algorithms thus inducing new biases. In this work we focus on the recurrent problem of duplicated records that, if not handled properly, may cause over-optimistic estimations of a model's performances. We introduce and discuss stratified cross-validation, a validation methodology that leverages stratification techniques to prevent data leakage in federated learning settings without relying on demanding deduplication algorithms.
△ Less
Submitted 23 January, 2020; v1 submitted 22 January, 2020;
originally announced January 2020.
-
A novel approach for identifying and addressing case-mix heterogeneity in individual participant data meta-analysis
Authors:
Tat-Thang Vo,
Raphael Porcher,
Anna Chaimani,
Stijn Vansteelandt
Abstract:
Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach…
▽ More
Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all trials to infer the treatment effect for the patient population in a given trial, based on direct standardization using either outcome regression (OCR) or inverse probability weighting (IPW). Accompanying random-effect meta-analysis models are developed. The new approach enables disentangling heterogeneity due to case mix from that due to beyond case-mix reasons.
△ Less
Submitted 24 August, 2021; v1 submitted 28 August, 2019;
originally announced August 2019.
