Limitations of models for guiding policy in the COVID-19 pandemic

At the outset of the COVID-19 epidemic in the UK, infectious disease modellers advised the government that unless a lockdown was imposed, most of the population would be infected within a few months and critical care capacity would be overwhelmed. This paper investigates the quantitative arguments underlying these predictions, and draws lessons for future policy. The modellers assumed that within age bands all individuals were equally susceptible and equally connected, leading to predictions that more than 80% of the population would be infected in the first wave of an unmitigated epidemic. Models that relax this unrealistic assumption to allow for selective removal of the most susceptible and connected individuals predict much smaller epidemic sizes. In most European countries no more than 10% of the population was infected in the first wave, irrespective of what restrictions were imposed. The modellers assumed that about 2% of those infected would require critical care, far higher than the proportion who entered critical care in the first wave, and failed to identify the key role of nosocomial transmission in overloading health systems. Model-based forecasts that only a lockdown could suppress the epidemic relied on a survey of contact rates in 2006, with no information on the types of contact most relevant to aerosol transmission or on heterogeneity of contact rates. In future epidemics, modellers should communicate the uncertainties associated with their assumptions and data, especially when these models are used to recommend policies that have high societal costs and are hard to reverse. Recognition of the gap between models and reality also implies a need to rebalance in favour of greater reliance on rapid studies of real-world transmission, robust model criticism, and acceptance that when measurements contradict model predictions it is the model that needs to be changed.