In August, thousands attended the Sturgis Motorcycle Rally in South Dakota for ten days of “riding, food, and music.” Approximately, 395,453 vehicles passed through — a 7.5 percent decrease from 2019.
While the cause of the decrease can’t officially be explained, it was going on during the coronavirus pandemic. As Smash Mouth lead vocalist Steve Harwell noted in their concert during the Sturgis rally, “Fuck that Covid shit.”
Harwell’s statement is the opening salvo of a disputed report on the Sturgis rally branding it a “superspreader” event. Released this week by San Diego State University’s Center for Health Economics & Policy, the report, which is not peer-reviewed, estimates that 266,796 new Covid-19 cases throughout America could be linked to the 460,000-person event.
It was an attention-grabbing data point, and the report, which published Saturday, quickly went viral. Initial reaction oscillated between some who described the attendees “stupid” and others who branded the numbers “fake news.”
From there, the controversy surrounding the report started to build. The estimate drew criticism from government officials and scientists. But what the report may truly represent, experts tell Inverse, is a fundamental misunderstanding between the public and researchers over how the process of scientific inquiry works, set against a backdrop of incredibly high stakes and slippery, uncertain data.
In the wake of the rally, the South Dakota Department of Health has only found 124 cases linked to the rally through contact tracing, while a survey of health departments by The Washington Post indicates that at least 260 cases across 11 states are connected to the rally.
Meanwhile, Governor Kristi Noem called the report “grossly misleading.” The rally’s organizers also put out a statement saying the report is “blatantly faulty.” Leading scientists, like Ashish Jha, dean of Brown University School of Public Health, say the report’s methodology didn’t “pass the sniff test.”
“Working papers” like this report, the equivalent of a “pre-print” paper in medical literature, are assumed to have some issues before they finish undergoing peer review. But the pressure of Covid-19 to publish research, and fast, has pushed researchers to present reports like this to the public as soon as possible. In the best case, the research helps people make better choices and guide further science. In the worst, it can erode public trust in science if the research is judged as flawed.
In between lies a situation that Yotam Ophir, assistant professor at the University at Buffalo who specializes in health and science communication, describes as “truly sad.”
“By not communicating the preliminary nature of the findings carefully enough, the scientists allowed the debate to shift away from what really matters — the fact that the rally was utterly and ethically irresponsible,” Ophir tells Inverse.
What the study claims, and what it can’t — The working paper took anonymized cell phone data and compared it to CDC case counts to show that, a month after the rally, Covid-19 cases in Meade County, increased by approximately 6 to 7 cases per 1,000. The city of Sturgis is located in Meade County, in western South Dakota. It also found the other counties which contributed the highest inflow of participants experienced a 7 to 12.5 percent increase in cases.
The report authors argue the rally resulted in a “superspreader” event, driving 266,796 new cases, and generating public health costs of $12.2 billion. This price tag is based on an estimated cost for non-fatal Covid-19 cases that are severe enough to require hospitalization of $46,000.
“… the scientists allowed the debate to shift away from what really matters — the fact that the rally was utterly and ethically irresponsible.”
There are two important points to note here. The first is that the report is primarily an economics paper, not a piece of epidemiological research. The second is that the actual cost of Covid-19 hospital stays can vary immensely, and the authors note that “this is by no means an accurate accounting of the true externality cost of the vent,” but argue it’s useful as a ballpark estimate of what superspreader events can cost.
Kevin Griffith is an assistant professor in health policy at Vanderbilt University School of Medicine. He was not involved in writing the working paper. He tells Inverse, that, in his assessment of the report, “Sturgis likely led to thousands of new cases and some deaths, but if the rally led to 250,000 new cases we would see it more clearly in the raw data.”
Ashley O’Donoghue, an economist at the Center for Healthcare Delivery Science at Beth Israel Deaconess Medical Center who recently co-authored the paper “Super-spreader business and risk of Covid-19 transmission,” offers a similar evaluation, drawing attention to issues with the researchers’ control group.
“It’s probably very likely that there was spread of Covid at the event based upon what we know about the transmission of the disease, but it’s probably less than 250,000 cases,” O’Donoghue tells Inverse. She was not involved in the report.
“I think that the way their control group is constructed leads me to believe that their results are an overestimate,” she says.