When you’re in a hurry, it’s much easier to make mistakes.
Admittedly, that’s a pretty mundane observation, but it seems to capture what a number of experts are saying about the recent rush to put AI into service in the battle against the coronavirus.
For example, as John Quackenbush, chair of the Department of Biostatistics at the Harvard T.H. Chan School of Public Health, observed, “I’ve heard a lot of hype about machine learning being applied to battling Covid-19, but I haven’t seen very many concrete examples where you could imagine in the short- or medium-term something that is going to have a substantial effect.”
To be sure, nobody is saying AI shouldn’t be tried, but it’s important to recognize the value of outcomes data that is reliable and can be tested. And while that’s tricky in the best of times, it’s particularly challenging in view of how new research into COVID-19 really is, and how limited, and still-unorganized, is the stockpile of relevant data.
“Everything that we’re doing gets better with a lot more well-annotated datasets,” said Dr. Eric Topol, director of the Scripps Research Translational Institute. “In the U.S., we don’t have centralized data. Here we are at the epicenter and all of our healthcare data is fragmented.”
On the other hand, as datasets get larger, they become “noisier.” For example, a model that screens Covid-19 patients for temperature might be reasonably effective. But expanded to the general population, “it’s a terrible predictor,” Quackenbush said.
Nonetheless, both men are “cautiously optimistic about using AI in some settings, such as determining which patients face a higher risk from Covid-19, opening an opportunity for communication with their physician.”
Still, the article notes some significant bumps in the road. For example, drugmaker Eli Lilly recently announced it would launch a trial of its existing rheumatoid arthritis treatment, baricitinib, in severely ill Covid-19 patients. But despite the reportedly successful identification of the drug’s potential by a British startup, BenevolentAI, “a group of rheumatologists that had treated patients in Lombardy, Italy, cautioned about potential adverse effects from the drug. Its FDA black box warning indicates patients taking the drug may face an increased risk of developing serious infections.”
Similarly, while AI is also being tapped to help with image recognition when it comes to detecting Covid-19 from CT scans, experts have raised a few concerns, including the fact that the CDC does not currently recommend using CT scans to diagnose Covid-19.
But there are no doubt valuable possibilities.
Said Quackenbush, “If we get to the point where we have access to the right data, and can really train good models to really understand which patients need to be monitored most closely, I think we’re in a good place to respond.”