COVID-19 pandemic surges can induce bias in trials using response adaptive randomization: A simulation study

Response-adaptive randomization is being used in COVID-19 trials, but it is unknown whether outcome rate changes during surges of COVID-19 will lead to bias in trial results. In response-adaptive randomization, allocation ratios are adjusted according to interim analyses to assign more patients to promising interventions. Although it is known that response-adaptive randomization may give biased estimates if outcome rates drift over time, observed mortality fluctuations in the COVID-19 pandemic are more extreme than any previously tested in simulation. We hypothesized that pandemic surges induce bias in trials using response-adaptive randomization, and that adjustment for time will alleviate that bias. Bayesian 4-arm superiority trials with a mortality outcome were simulated to investigate bias in treatment effect, comparing complete and response-adaptive randomization under different pandemic scenarios based on data from New York, Spain, and Italy. Relative bias in the odds ratio associated with treatment ranged from 0.3% to 11% and was largest in trials with a surge and an effective intervention that did not adjust for time. Bias was attenuated by adjustment for time without compromising the false-positive rate. Trials using response-adaptive randomization were more likely to identify effective interventions but were slower to drop ineffective interventions. Even with variation in outcome rates similar to observed pandemic surges, COVID-19 trials using response-adaptive randomization that adjust for time can provide accurate estimates of treatment effects.