How Models Were Used in the COVID-19 Pandemic
While mathematical models themselves have proved to be interesting to dissect, it’s really important to understand the process by which they are used in decision-making. The COVID-19 pandemic has documented this in great detail, and I’ll be sharing some peer-reviewed articles that might be particularly useful in understanding how models were used in the pandemic.
In April of 2020, Panovska-Griffiths asks the question “Can mathematical modeling solve the current Covid-19 crisis?” In this article, she explores the differences in the research questions, calibration, and interpretation between two models that contributed to the literature in informing United Kingdom policy at the very beginning of the pandemic. Both of these models were preprints and did not have time for peer-review due to the rapid response necessary at the time. The most important takeaway from Panovska-Griffiths is that focusing on which of these models is right in predicting the future is a misguided interpretation of them. They are merely trying to provide a framework for understanding the effects of interventions or lack thereof.
Note: there are models - forecasts - that are used to predict short-term outcomes. Here is some information on US-based COVID-19 forecasting efforts.
Metcalf et al. carefully explain data needed to inform models and the interventions explored so that modeling can be beneficial in understanding the effects of costly interventions (lockdowns, social distancing, etc.). Quantifying the relative impact of interventions is tricky, so the models that are calibrated with this type of data should be interpreted knowing the limitations of it. The authors also explain when different model structures might be required. For example, age stratification is important when considering the impact of schools closures but might not be necessary when understanding the impact of other interventions. Finally, they underscore that mathematical models should continue to be refined as they are evaluated retrospectively, as models are an important tools in understanding future scenarios.
Image from Metcalf et al.
Adiga et al. provide a more technical review of several COVID-19 models from 2020. They describe methodology in general for compartmental models, metapopulation models, agent-based models, and forecasting models. This paper points out the important details all readers of scientific articles should be looking out for. The authors compare prominent UK, US, and Swedish models to one another, laying out the strengths and weaknesses in both the assumptions and methodology. They state that comparing the projections in the compartmental and agent-based models to the actual outcomes is not useful because the aim of those models is the provide a relative understanding of what the impacts of specific decisions may be. On the other hand, forecasting models can be useful for predicting outcomes in the short term to aid with hospital capacity and resource questions. Adiga et al. disagree with Dr. Vikram Patel’s criticism of models that estimated “of mountains of dead bodies which fuelled the panic and led to the unprecedented restrictions on public life around the world” because those models were estimating COVID-related burden given that restrictions were not put into place. Also, Dr. Vikram Patel’s comments were in 2020, prior to some of the worst COVID-19 waves that occurred in 2021 around the globe and did result in a rapid increase in COVID-related deaths.
Finally, I wanted to take a look at Biggerstaff et al. The authors lay out the process by which the US Centers for Disease Control and Prevention used modeling to evaluate intervention strategies during the COVID-19 pandemic. The authors compile models from both 2020 and 2021 that informed pandemic guidance. The previous papers review the effects of individual models on policy. And while Biggerstaff et al. highlight the CDC models, they actually emphasize the need for collaboration across the agency but also across academic and industry partners. Multi-model approaches were pertinent to forming nuanced interventions.
Overall, these four papers provide context for the effects of modeling in the western world during the COVID-19 pandemic. These are the frameworks we should be using to evaluate individual models and support the need for modeling collaboration so that multiple models can be used to form more thoughtful conclusions in the future. The answer to Panovska-Griffiths' question is that models can be useful in guiding pandemic response when officials consider both the objectives and limitations of individual models and consult multiple models to inform the policies in question.
References:
Panovska-Griffiths J. Can mathematical modelling solve the current Covid-19 crisis?. BMC Public Health. 2020;20(1):551. Published 2020 Apr 24. doi:10.1186/s12889-020-08671-z
Metcalf CJE, Morris DH, Park SW. Mathematical models to guide pandemic response. Science. 2020;369(6502):368-369. doi:10.1126/science.abd1668
Adiga A, Dubhashi D, Lewis B, Marathe M, Venkatramanan S, Vullikanti A. Mathematical Models for COVID-19 Pandemic: A Comparative Analysis. J Indian Inst Sci. 2020;100(4):793-807. doi:10.1007/s41745-020-00200-6
Biggerstaff M, Slayton RB, Johansson MA, Butler JC. Improving Pandemic Response: Employing Mathematical Modeling to Confront Coronavirus Disease 2019. Clin Infect Dis. 2022;74(5):913-917. doi:10.1093/cid/ciab673