Validation and Sensitivity Analysis of the COVID-19 Transmission Model Simulating Counterfactual Infections in Japan

BackgroundKayano et al. estimated the potential number of COVID-19 cases and deaths in Japan from February 17 to November 30, 2021, in a counterfactual scenario where no vaccination was implemented. Their model predicted 63.3 million cases and 364,000 deaths, with a claimed 95% confidence interval of less than 1% of the estimated values. ObjectiveTo validate the transmission model used by Kayano et al. by simulating infection counts in Japan during 2020 and assessing the impact of errors in the reproduction number of the Delta variant on infection count estimates in the counterfactual scenario. MethodsWe replicated the model used by Kayano et al. to simulate infection dynamics in Japan during 2020. We evaluated the models performance by comparing the simulated infection surges with actual data. Additionally, we analyzed the sensitivity of infection count estimates to {+/-}10% errors in the reproduction number of the Delta variant, corresponding to the 95% confidence interval reported in the cited study. ResultsThe model successfully reproduced the first infection surge in early 2020, but it failed to replicate the second and third infection surges in the summer and winter of 2020. Sensitivity analysis revealed that a {+/-}10% error in the reproduction number led to an estimated infection count error range of -25% to +42%. ConclusionThe results suggest potential limitations in the validity of the counterfactual simulation results presented by Kayano et al., particularly regarding the accuracy of infection count estimates and the confidence interval.