The webinar Hybrid Dynamic Models in COVID-19 Planning and Beyond drew in a record number of attendees and prompted some great questions around the theme of simulation modeling and healthcare planning.
We had the opportunity to host the webinar with Dr. Nathaniel Osgood, a Professor in the Department of Computer Science and Associate Faculty in the Department of Community Health & Epidemiology at the University of Saskatchewan. With so many attendees and such a relevant topic, given the current situation, we experienced lots of interaction and great questions for our webinar panelists! Below, you will find the webinar details, the recording, and Dr. Nathaniel Osgood’s Q&A answers.
Hybrid Dynamic Models in COVID-19 Planning and Beyond
While the various traditions of dynamic modeling have each conferred much insight for policy analysts and decision-makers in public health and health care, the potential of each is limited by a set of characteristic limits. More broadly, once built, models from each dynamic modeling tradition have traditionally suffered from rapid obsolescence, becoming increasingly disconnected from new evidence and unfolding of processes in the world treated as stochastically evolving in the model.
Aided by machine learning tools — including algorithms such as Particle Filtering and Particle MCMC — many elements of the above learning process can be automated, incorporating new incoming evidence on a frequent ongoing basis, and using it to refine understanding of the latent state of the system, estimates of parameters playing an important role in driving the system, and evaluate policy trade-offs. Within this talk, we discuss how classes of hybrid models can aid in more insightful, transparent, and nimble modeling in the context of unfolding uncertainties, with the COVID-19 outbreak serving as a central exemplar.
Answers to the questions that were asked of Dr. Nathanial Osgood during the webinar
What do you think the impact would be of requiring visitors to have flu vaccine in aged care visits?
This is a very interesting question and could be very important for the risk of a 2nd or 3rd wave during the coming influenza season, including for distinguishing likely flu from covid.
Out of your calculations, what could be the best estimate for COVID19's "Ro" reproductive number in your opinion?
This varies notably by context even in our province – for example, in the context of crowded housing in the northern parts of our province, compared to rural or urban areas.
How does your approach to validation, calibration and sensitivity analysis differ in a hybrid model?
We can often undertake such tasks using data from different scales. It can also sometimes be helpful to engage in “partial model calibration” by swapping modules out and back in.
How do you integrate your models with different formalisms together such that they feed each other?
This is fairly straightforward in AnyLogic. Please see my many videos on the mechanics of building different types of Hybrid Models in AnyLogic.
In the context of Latin American countries and their handling of covid-19, do you think there are other variables that should be added to the model? If so, what are they?
To properly answer this question, I would need to learn more about the Latin American context, including public transit, residential mixing, crowding and disparities therein, indoor vs. Outdoor working environments.
The number of unreported cases is a key challenges for policy makers and care providers. What insights did you surface with your model in this regard? Can it back-cast the portion of unreported cases?
Yes, it can back-cast number of unreported cases, effective reproductive number, estimate of care-seeking behavior, etc.
What is the performance of the particle model when no confirmed cases are available?
This case serves as a very important and successful application of the particle filtering model, as it tends to reground the model in a way that weeds out/downweights particles positing outbreaks during this time, and maintains the set of susceptible until and outbreak does occur.
Could you please explain in more detail the way that the model accounts for social distancing and the government's stay-at-home orders?
This is complex, as it broadly affects workplace, school, home and community mixing. Social distancing affects the balance of time that they spend in the community. Work in the near future will be representing community context with greater granularity.
Are all patients in the DES just within the hospital or does the pathway go in the community?
Some of the DES component can include community components. Agents remain individuated when leaving the pathway.
Can your PF model incorporate data from different countries/provinces to learn from and/or compare them?
The PF model has been successfully applied in a variety of distributions. By contract from Public Health Agency, we are currently applying it across Canadian provinces, and have received requests for applying it to other countries as well.
In your model, can severe cases turn into critical cases? If so, how can it affect the model?
For one of the models shown, “severe” includes a subset of critical cases (as defined by the WHO). In the ABM, critical cases explicitly considered.
Are economic aspects (hospital funding etc.) also considered in the model that is presented today?
Currently, cost is not considered directly, although resource use serves as a rough proxy. We are planning to add costs.
Is particle filtering done from within AnyLogic?
Yes, the particle filtering is conducted within AnyLogic. While our methods have advanced much since then, the basic of the methods are outlined in Example AnyLogic Simulation Model with Particle Filtering. [YouTube video]
What is the CI of particle result?
Being a Bayesian method, the CI is a Credibility Interval here. Particle filtering estimates (by sampling from) a joint distribution over the model states; we readily compute the credibility intervals.
How is overall error calculated via heterogenous sources of model data?
Particle filtering uses an overall likelihood function that place a key role in updating the prior to the posterior with each new vector of observation. The likelihood function takes into account the different types of observations.
In the hybrid agent based, system dynamics model, what size of population can you accommodate given that a very large percentage will eventually become an agent? Also is there a way to capture age related transmission before creating the agents?
Yes, the aggregate transmission model is age stratified – and is in the process of further stratification for chronic disease. Based on other models from our group, we expect to be able to handle hundreds of thousands of agents.
If I would be interested to work together, what is the first step to get into touch?
We would be open to exploring collaboration. Several of our models are being adopted and adapted elsewhere. Knowing which model(s) were being considered would be helpful to advise. Please write to christine.hillis@usask.ca if you’d like to set up a meeting to explore this.
Are the models and/or documentation publicly accessible to review?
Public release is anticipated but requires finalization with our health system partners. Please write if you’d like to be notified.
Could we be emailed a linked to the presenter's homepage, and also to any papers (working or published) that he has written on this?
Nathaniel Osgood Faculty Homepage. Many papers can be found on this [Moving Beyond Blindfolded Models page]
Is there any possibility for research collaboration if we want to apply the model in Australian health sector?
We would be open to exploring collaboration. Several of our models are being adopted and adapted elsewhere. Knowing which model(s) were being considered would be helpful to advise. Please write to christine.hillis@usask.ca if you’d like to set up a meeting to explore this.
Are the example models available for download?
Public release is anticipated but requires finalization with our health system partners. Please write if you’d like to be notified.
Is your bootcamp (8/24~8/29) going to be held in person or online this year?
Thanks for the question, due to the COVID-19 outbreak, we are not able to hold the bootcamp in person this summer. We are considering holding an online version, but would need serious expression of interest. Please write to christine.hillis@usask.ca if you are interested.
Can we run your SD/ABM/DES for other provinces in Canada? E.g., PEI
We would be open to exploring collaboration. Several of our models are being adopted and adapted elsewhere. Knowing which model(s) were being considered would be helpful to advise. Please write to christine.hillis@usask.ca if you’d like to set up a meeting to explore this.
About Dr Nathaniel D. Osgood
Dr. Nathaniel D. Osgood is a Professor in the Department of Computer Science and Associate Faculty in the Department of Community Health & Epidemiology at the University of Saskatchewan. His research is focused on providing cross-linked simulation, ubiquitous sensing, and machine learning tools to inform understanding of population health trends and health policy tradeoffs. His applications work has addressed challenges in the communicable, zoonotic, environmental, and chronic disease areas. Dr. Osgood is further the co-creator of two novel mobile sensor-based epidemiological monitoring systems, most recently the Google Android- and iPhone-based iEpi (now Ethica Health) mobile epidemiological monitoring systems. He has additionally contributed innovations to improve dynamic modeling quality and efficiency, introduced novel techniques hybridizing multiple simulation approaches and simulation models with decision analysis tools, and which leverage such models using data gathered from wireless epidemiological monitoring systems. Dr. Osgood has led many international courses in simulation modeling and health around the world, and his online videos on the subject attract thousands of views per month. Prior to joining the U of S faculty, he graduated from MIT with a PhD in Computer Science in 1999, served as a Senior Lecturer at MIT and worked for a number of years in a variety of academic, consulting and industry positions.
