Reweighting the RCT for generalization: finite sample analysis and variable selection
share
The limited scope of Randomized Controlled Trials (RCT) is increasingly under scrutiny, in particular when samples are unrepresentative. Indeed, some RCTs over- or under- sample individuals with certain characteristics compared to the target population, for which one want to draw conclusions on treatment effectiveness. Re-weighting trial individuals to match the target population helps to improve the treatment effect estimation. Such procedures require an estimation of the ratio of the two densities (trial and target distributions). In this work, we establish the exact expressions of the bias and variance of such reweighting procedures – also called Inverse Propensity of Sampling Weighting (IPSW) – in presence of categorical covariates for any sample size. Such results allow us to compare the theoretical performance of different versions of IPSW estimates. Besides, our results show how the performance (bias, variance and quadratic risk) of IPSW estimates depends on the two sample sizes (RCT and target population). A by-product of our work is the proof of consistency of IPSW estimates. A refined analysis also shows that IPSW performances are improved when the trial probability to be treated is estimated (rather than using its oracle counterpart). In addition, we study how including covariates that are unnecessary to a proper estimation of the weights may impact the asymptotic variance. We illustrate all the takeaways twice: in a toy and didactic example, and on a semi-synthetic simulation inspired from critical care medicine.
READ FULL TEXT