Switchback Experiments under Geometric Mixing
share
The switchback is an experimental design that measures treatment effects by repeatedly turning an intervention on and off for a whole system. Switchback experiments are a robust way to overcome cross-unit spillover effects; however, they are vulnerable to bias from temporal carryovers. In this paper, we consider properties of switchback experiments in Markovian systems that mix at a geometric rate. We find that, in this setting, standard switchback designs suffer considerably from carryover bias: Their estimation error decays as T^-1/3 in terms of the experiment horizon T, whereas in the absence of carryovers a faster rate of T^-1/2 would have been possible. We also show, however, that judicious use of burn-in periods can considerably improve the situation, and enables errors that decay as log(T)^1/2T^-1/2. Our formal results are mirrored in an empirical evaluation.
READ FULL TEXT