Administration and Leadership, Coronavirus, Exclusives

Unsettled Science: Statistics and Government Decision-making in the Era of COVID-19

FEMA logistics specialists and New Jersey National Guardsmen help unload Battelle decontamination units at the New Jersey Exposition Center in April 2020. (FEMA Photo/K.C. Wilsey)

Public health usually bubbles along behind the scenes of government policy as a clearly needed but generally less-than-urgent aspect of public administration. To the casual observer, it often focuses addressing slow moving crises like obesity or smoking. Over the past several months, though, the immediacy of the connection between public health and government policy has become readily apparent to everyone. COVID-19 has forced politicians and other decision makers (as well as journalists) into the unfamiliar world of biostatistics, often with rocky results. In many cases (like my own), it has also put emergency managers, public health professionals, and local prehospital emergency medical services personnel in the position of serving as a kind of ad hoc translator of the complex data that decision makers are relying on for day-to-day decisions. 

This sudden baptism of policy makers into the public health world has resulted in a fair amount of discussion about politicians misrepresenting COVID-19 statistics. A balanced perspective would see that public health issues always involve some amount of politics, but the immediacy and host of unknowns involved in the emergency of SARS-COV-2 have highlighted the differences among public health data, medical science, public policy, and journalism/opinion, often resulting in decision makers and medical professionals talking past each other and commentators of all political stripes pouncing on any discrepancies. Some have argued that the numbers of COVID deaths and cases have been blown out of proportion to make the pandemic look much worse than it in a quest to increase government’s scope or help the president’s opposition.1 Conversely, the early-opening states have been accused of manipulating the data to justify their purportedly reckless policies.2 There is a lot of room for finger pointing in a crisis as fast moving and variable as this one.

Related

We will eventually discover the fountain of truth in the tidal wave COVID-19’s statistics we all are drowning in right now. Some things will get nailed down sooner rather than later. For example, we should have a good idea by midsummer if Georgia is truly experimenting in human sacrifice by opening nail salons (according to an article in The Atlantic)3 or if Michigan’s stay at home orders have significant unintended consequences (including an economic downside with its own non-pandemic public policy concerns).4 But the nation also needs to be prepared for the fact that it may be months or years before the number crunchers sort the whole statistical mess out. Unfortunately, that time is not on the side of our decision makers right now.

In a posting on the New England Journal of Medicine, Dr. Eric Schneider asserts that “without good testing data, forecasters have to rely on guesswork and assumptions.” Fatality estimates have ranged from thousands to millions and researchers are only beginning to identify key information like the virus’ transmissibility, its lethality, and what it does to the human body. This makes predictive modeling very difficult. To put it in Schneider’s terms, it is “problematic” that there is “a paucity of the facts required to inform models.”5

If COVID-19’s data is problematic for medical professionals, this barrage of inconsistent information has been downright impossible for the decision makers. The truth is that COVID-19 statistics are all over the place right now. The Institute for Health Metrics and Evaluation (IHME) models have come under scrutiny for their ever-changing numbers even while they try to give hospitals a functional planning template.6 While the IHME has gotten a lot of attention (and figured prominently in the White House Coronavirus Task Force briefings during the height of the spring 2020 shutdowns), many public health research entities are working full tilt to understand the virus, from Harvard’s TH Chan School of Public Health and Emory University in Atlanta, to the World Health Organization and hospitals around the world.

In the midst of scientific inquiry, the sneak peaks we get of the medical community’s progress is often disjointed and confusing and can be used to logically support an array of policy decisions. (There is even a website dedicated to maintaining records of how the projections of the various COVID-19 models have changed over the course of the virus.7) They are working with the same scarcity of facts that we all are. Similarly, states are trying to rapidly assemble and assess data from coroners, hospitals, public health agencies, and nursing homes and meet rapid deadlines that ensure statistical errors and revisions. On top of this, half of America is now a budding epidemiologist and is scrutinizing these public health professionals in real time. Their results are then getting turned into policy action before the ink on their research papers is even dry.

In this shifting and limited data climate of COVID-19, governors, mayors, the White House, and appointed officials of all stripes are trying to chart a course among the perilously abutting concerns of health, civil rights, the economy and politics.  They are all on the hunt for data—for some type of settled science—to support making the right decision.

Unfortunately for these decision-makers, science and policy don’t always operate on the same timeline. Science is an iterative process–it usually takes multiple steps, dead-ends, recalibration and challenges to confirm anything with any certainty. It involves someone presenting an idea based on research and statistics then subjecting it to peer review and—quite often—heavy criticism, which results in correction and thereby progress. Science is, therefore, inherently unsettled and relies on challenge to move forward over time. Policy, unfortunately, likes settled science as a foundation for solid (and politically safe) decision making. 

In the current pandemic, though, the American public—and the world—is impatiently watching the scientific method unfold real-time. Transmissibility, mortality, disease presentation, the impacts of drugs and the benefit of lockdowns are still up for debate. Researchers can make educated guesses from past experience like they did with Remdesivir,8 but despite recent hopeful signs, more research still needs to be done to fully understand its impact. 

For the policy makers, the science of the novel coronavirus could be said to be on everybody’s side right now (or nobody’s). Time will tell which state has charted the best course during this unprecedented event, but it is a bit premature to definitively declare whose side the science is on right now. Over the next several months (or years), there are going to be many times when the data and the public policy don’t mesh. When discrepancies crop up between datasets or among policies, we need to work together to find ways to overcome them. Instead of understanding and cooperation, though, this process has currently devolved into something of a political blood sport. 

The only cure for this policy-science divide is for the public health and emergency medical services community to mount an educational outreach campaign to provide perspective to the debate. For us, public health is an applied field—we practice in overturned cars and living rooms and on street corners across the nation. We also have some understanding of the studies and statistical implications of the COVID-19 research being conducted right now. The EMS community needs to be the guide to local governments and state policy makers through the complexities of the pandemic. While decision makers like to have a settled scientific foundation to their policy decisions, we need to be able to explain that the COVID-19 research is still ongoing—the science is still very unsettled in this critical area—and local government policy will have to keep this in mind for the foreseeable future. This awareness will result in better policies from our decision makers and, hopefully, better outcomes for our communities.

References

1. Hawkins, Derek. Eric Trump claims coronavirus is Democratic hoax, will ‘magically’ vanish after 2020 election. Washington Post. May 17, 2020. https://www.washingtonpost.com/politics/2020/05/17/eric-trump-coronavirus/.

2. Mull, Amanda. Georgia’s Experiment in Human Sacrifice: The state is about to find out how many people need to lose their lives to shore up the economy. The Atlantic. April 29, 2020.  Retrieved from https://www.theatlantic.com/health/archive/2020/04/why-georgia-reopening-coronavirus-pandemic/610882/.

3. Ibid.

4. McClallen, Scott. Business groups highlight possible economic impact of a ‘stay-at-home’ order. Washington Examiner. March 23, 2020.  https://www.washingtonexaminer.com/politics/business-groups-highlight-possible-economic-impact-of-a-stay-at-home-order.

5. Schneider, E.C. Failing the Test – The Tragic Data Gap Undermining the US Pandemic Response. The New England Journal of Medicine. May 15, 2020. https://www.nejm.org/doi/full/10.1056/NEJMp2014836?query=featured_home.

6. Begley, Sharon. Influential Covid-19 model uses flawed methods and shouldn’t guide U.S. policies, critics say. Stat News. April 17, 2020. https://www.statnews.com/2020/04/17/influential-covid-19-model-uses-flawed-methods-shouldnt-guide-policies-critics-say/.

7. COVID Projections Tracker [Internet]. [cited 2020 Jul 7]. Available from: https://www.covid-projections.com/.

8. Gilead. About Remdesivir. Retrieved from Gilead: Creating Possible. May 20, 2020. https://www.gilead.com/purpose/advancing-global-health/covid-19/about-remdesivir