Methods for Reproducible Comparison of Strategies in Stochastic Modelling

Stochastic simulations are often used to model real-world phenomena such as infectious disease dynamics. In this modelling, differing strategies are often compared to one another by comparing the model outputs each strategy results in. Hash-based matching, pseudo-random number generation is an approach for stochastic simulations that was originally developed by Pearson and Abbott in the hashprng package to overcome challenges with comparing model simulations in a way that considers the dependency between model outputs. We demonstrate how methods based on this approach grant considerable benefit when comparing different strategies, and show when each of our three proposed methods ought to be used. We illustrate our methods with two epidemiological models: one simple model of a vaccine-preventable infection and one complex model of African sleeping sickness, which can be controlled through multiple interventions. We show how our Bernoulli hashing method works very well for simple models, and a variation of it can be used for more complex models in certain cases. Additionally, we discuss the properties of our methods for considering counterfactual scenarios and note that, compared to other attempts to obtain perfect counterfactuals, they demonstrate advantages in computational complexity and their application to a wider variety of models.