FSCJ Data Scientist Says The Way Fla. Presents COVID-19 Numbers Is Misleading

A Jacksonville-based data scientist says the way Florida presents COVID-19 data to the public is misleading, and he worries it’s being done deliberately to help support efforts to reopen the state.

Christopher Perle is a professor of biology at Florida State College at Jacksonville. He has a Ph.D. from Stanford University, where he frequently dealt with enormous amounts of data, and has been collecting and reporting data for over 25 years - primarily related to biostatistics, marine ecology and oceanography.

He’s also taught graduate and undergraduate-level classes in biostatistics, including interpretation of large datasets.

Perle has been keeping a close eye on Florida’s COVID-19 dashboard since it first launched.

“I had some concerns from the get go, and I've only got more concerns now,” he said.

And Perle isn’t the only expert who thinks the state deserves some scrutiny over how it’s presenting data.

“I think Dr. Perle is bringing up very, very good questions,” said Aditya Khanna, assistant professor of research in the Department of Medicine at the University of Chicago.

One of Perle’s main concerns is related to the way that positive cases and deaths are reported.

The Florida Department of Health regularly releases updates on COVID-19 with numbers from throughout the state. But, it frequently takes some time to get positive test results back or confirm the date of death. 

When the state gets confirmation, it retroactively adds those numbers to its dashboard, sometimes as many as six weeks after the fact. This means that the new cases by day and date of death graphs almost always appear to be trending downwards, unless there is a very strong upward trend like the state is seeing now with positive cases.

State health officials say numbers are rising because more tests are being conducted, but an analysis from The Miami Herald found that the recent increase in new cases can’t be explained by increased testing alone. That analysis found that the 64 counties that moved into the second phase of reopening on June 5 saw close to a 42% increase in new cases the week before, while testing had increased by 8% over the same period.

Meanwhile, a ProPublica report found that positive tests per 100,000 people and the percentage of tests yielding positive results are both increasing in Florida.

Related: Florida And S.C. Report New Spikes In Coronavirus Cases

“Maybe they have a good reason for doing that. What that reason might be is not really obvious to me,” Khanna said of the way the state presents new cases and date of death data. “I think if they choose to do it that way, for some reason, then it needs to be made crystal clear that the data reports that are being displayed are not final, they are subject to change, and almost certainly will change based on new reports that come in.”

The “Resident Deaths by Date of Death” graph on the state’s dashboard does include the qualifier “Death data often has significant delays in reporting, so data within the past two weeks will be updated frequently.” However, that is the smallest text on the page, and it may not initially be visible depending on the size of the window where the dashboard is viewed. It doesn’t appear to be on the mobile version of the site, and there is no similar warning associated with the “New Cases of Residents by Day.”

“You can't just throw me a snapshot and say this is what it is, because it's going to change tomorrow,” Perle said.

“You need to see the trend,” he explained. “Just talking about a particular day's value is not very meaningful if you're not talking about what those values have been for the last week or two weeks, or since major decision points.”

Florida data of death animation showing data being added retroactively. Credit: Christopher Perle

For example, during much of April, Duval County saw a pretty steady decrease in its seven-day rolling average of total COVID-19 cases and percentage of positive cases, while death totals and hospitalizations remained fairly level. That trend continued after Jacksonville Mayor Lenny Curry reopened the beaches on April 17. 

But all of those figures have started trending upward since Phase 1 of reopening began in Jacksonville, and around the state, on May 4, with record spikes in cases over the past week.

Another problem Perle has with the dashboard is that it is only reporting Florida residents. 

“There’s no reason for it just to be residents,” he said. 

“There are a number of people - perhaps in the more susceptible demographic groups, retired folks, elderly folks, so on - who go there [Florida] and spend a good deal of time in the winter, exactly when COVID was starting to become recognized as an issue. So I think that’s a very important question,” Khanna said.

However, the infection rate among non-residents is reported in the county-by-county breakdowns on the state dashboard.

In an email to WJCT, a Florida DOH spokesperson wrote, “Reporting deaths by residency is the appropriate method utilized to calculate disease rates, which allow for a more accurate analysis of disease impacts on populations through the incorporation of demographic data - a critical aspect of public health planning. Population size is determined using census data, which is based on residency.”

“We report data regarding Florida residents and non-Florida residents separately. Deaths are reported according to their place of residence. This keeps them from being inadvertently listed twice and is why only Florida residents are listed in Florida deaths. The state reports non-Florida deaths to the individual’s place of residency. Florida provides aggregate level data of non-Florida resident deaths to the CDC,” the DOH spokesperson went on to write.

Perle also takes issue with how test results from asymptomatic people are being lumped in with results from those who are showing symptoms, pointing to Jacksonville testing most of the city’s homeless population.

“They tested 700 homeless people who didn't have symptoms, who turned out to be all negative. It will make your percent positive go down when you start adding asymptomatic people,” he said.

Jacksonville Mayor Lenny Curry and Florida Gov. Ron DeSantis both frequently point to low and/or declining percent positive rates. 

When testing first began, there were very strict requirements that people had to meet to get tested. 

Initially, for a patient to qualify for testing they had to have a fever and be showing symptoms of a lower respiratory illness (like a cough or shortness of breath) and they must have traveled from a coronavirus hot spot (China, South Korea, Iran, Italy, and Japan at the time) within the past two weeks or they had to be able to prove that they had come in close contact with a confirmed COVID-19 patient.

Even people who were symptomatic couldn’t get tested if they didn’t meet those requirements.

The state started easing testing criteria in early-to-mid April, and has since tried to make testing as widely available as possible.

But Perle says it’s problematic to lump results from symptomatic and asymptomatic people together. 

“Percent positive will go down as percent asymptomatic goes up,” he said. “You can't really interpret that value [percent positive] without knowing who was asymptomatic and who wasn't. You’ve got to control for that, or you can very easily manipulate the percent positive figure based on who you test.”

Khanna said the easy solution would be to just present separate results from symptomatic and asymptomatic populations, that way the “numbers won’t be mixed up with each other.”

Others have expressed concerns over how the state calculates percent positive as well.

Rebekah Jones, who was fired in May from her job at the florida Department of Health, says on the state’s COVID-19 dashboard, which she helped create, any person who tests positive will be counted as a positive test only once, no matter how many times they test positive. But if a person tests negative, their results will be counted each time they test negative for the coronavirus.

“They’re adding their total test figures instead of their total people, which makes their percent positive extremely low,” Jones told NPR.

Since her firing, Jones has created her own dashboard, which looks very similar to the state’s at first glance but differs in several key ways. For example, as of Tuesday afternoon the state’s dashboard says 1,461,297 people have been tested in Florida, but Jones’ dashboard indicates that 1,093,427 have been tested.

According to the state’s dashboard, Duval County has a 2.9% overall percent positive rate. Jones’ dashboard, on the other hand, shows a 4.4% positive rate.

Related: As COVID-19 Cases Rise, Mayor Curry Says Jacksonville Has 'Flattened The Curve'

Jones’ dashboard also includes a map that indicates only one of Florida’s 67 counties (Liberty) is ready for the next phase of reopening. Jones claims she was fired from her job at the state DOH because she refused to manipulate data to make Florida appear as if it had met reopening criteria. Gov. Ron DeSantis and state officials have said Jones was fired for “insubordination.”

Khanna said, in his experience, there isn’t always malintent in the design choices somebody makes, and given his limited knowledge of Florida, he wouldn’t make such a claim.

“But, certainly impressions can be informed by a pattern or history of discussions around data and assessments,” he said. “Whether it's coming from the government, a private industry, scientific institutions, or academic institutions, I think design choices in the display and analysis of data are always legitimate areas to have questions about, and it should come as no surprise to somebody that those design choices are being questioned or require further clarification.”

Perle agrees that mistakes can be made honestly. But he thinks the state has had plenty of time to correct what he sees as its mistakes in this case, leading him to wonder if the state has deliberately presented the data in what he describes as a “shady” way to “support a particular narrative,” which, in this case, is that Florida has met reopening criteria.

Brendan Rivers can be reached at brivers@wjct.org, 904-358-6396 or on Twitter at @BrendanRivers.