Doing predictive analytics? You might be doing it wrong

'Don't try to predict everything under the sun'
By Mike Miliard
11:19 AM
Opposing sides

There are "two camps" when it comes to appreciating the power of predictive analytics in healthcare, says Sriram Vishwanath, professor of data science and informatics at University of Texas, Austin's Cockrell School of Engineering.

"One says, 'Anyone who's trying to do it must not be doing a good job. Healthcare is a hard thing and most people don't know what they're doing; descriptive is all you're going to get, and predictive is never going to be actionable – there are too many variables and it's never going to work," says Vishwanath by phone from the Silicon Valley area.

"The other camp, the data science camp, says 'I can do anything,'" he adds. "They tend to be from this area of California. They say, 'I can predict anything. Give me a challenge and I'll do it.'"

Unfortunately, "both camps are wrong."

That overconfident data scientist "doesn't know how hard this is," says Vishwanath. Meanwhile, the change-resistant clinician doesn't appreciate how data science, when shrewdly applied, can positively impact care delivery.

"There's a happy medium," he says. "Don't try to predict everything under the sun. Focus."

But that's only the most basic piece of advice for those looking to get going with big data. At the Healthcare IT News Big Data and Healthcare Analytics Forum in Boston this November, Vishwanath will draw on his experience in data science and engineering to offer some real world strategies for health organizations – providers and payers alike – looking to make some targeted gains with high-precision analytics.

As value-based care increasingly blurs the dividing line between payers and providers, he'll show how judiciously applied analytics can help improve both care management and overall risk management.

"Medical and financial risk are two different things, but in prediction they're heavily coupled," says Vishwanath. "It's two sides of the same coin."

Very soon, he says, "it's going to be irrelevant what side you're on. Increasingly, providers are taking financial risks. CMS is forcing them to. Health plans have already been taking risks, and with the acquisitions they're making on the provider side they're taking medical and financial risks."

[Learn more: Meet the speakers at the Big Data and Healthcare Analytics Forum.]

On both sides, he says, the goals are pretty much the same: "spot patients, physicians and facilities where proactive engagement and targeted interventions can significantly reduce costs while improving health-outcomes."

Professor at UT for more than a dozen years, and recent CEO of health analytics startup Accordion Health, Vishwanath had been swimming in data analytics for a while before he realized what unique challenges were faced by healthcare: "pain points from a data science perspective, in terms of operationalizing and understanding from a clinical and financial perspective how predictive analytics could really impact care," he says.

"I've been doing predictive analytics for many many years. First in a network setting, network traffic, then I did consumer analytics, understanding consumer behavior, how consumer trends change over time," says Vishwanath.

One thing he noticed when he really set his sights on healthcare was that it was "changing rapidly compared to the way it had been." With new troves of data that simply didn't exist in the paper-based era, the industry is finally poised for big improvements in quality and cost.

It's the "right time," he says. "It could really add value."

Nonetheless, Vishwanath also soon noticed "a fairly large gap between those who said they did prediction and those who actually did prediction. The term 'predictive analytics' was thrown around – similar to how 'AI' is thrown around and 'synergy' is thrown around. It was a buzzword. When you spent some amount of time going into the backside and saying, 'What exactly is it that you're doing?' and asking some hard questions, at that point it became quite apparent that they were not predicting. They were summarizing, they were interpolating, they were extrapolating."

He likes to illustrate the perils of a facile approach to predictive data with one of his favorite xkcd comics:

It just doesn't quite work that way.

Healthcare has been making some big advances in predictive analytics recently, which is in some ways more impressive because of the disadvantage it's at compared with other industries.

"Data scientists are so used to consumer data," says Vishwanath. "Consumer data is easy, compared to healthcare data: Each consumer tells you so much about them it is mind-boggling how much data there is. Every consumer you end up with about 100,000 points. Gobs of data."

In healthcare, on the other hand, "it's still a big data problem, but on every individual trajectory you want to study, you have maybe three or four data points. If you're lucky, 10 or 12 – and that's not lucky, because that person is an over-utilizer already and predicting them is easy."

The challenge we're all after, of course, "is predicting someone who's been in the system two or three times, and you're predicting this person is on the trajectory to show up another 20 times," he says. "That's the hard part."

Healthcare, says Vishwanath, "has a little data/big data problem: On every row there are two or three data points, but the number of rows is tens of thousands. Through three points you could stretch a gazillion curves."

That's why it's important to focus.

"You predict on a piece of data," he says. "When someone gives you a predictive model, they're not giving you a dashboard, a bunch of pretty pictures, they're giving you something they've validated on data. You gave them data, they hid six months from themselves, and predicted that six months correctly – It's very easy to check if it works. If it's validated, then it's valuable. If it's not validated, it's smoke and mirrors."

The lesson? "Predict what you're good at." For other contingencies, "say, 'Sorry, no can do.' That's the right mindset for healthcare."

For example, he says, "predicting cardiac arrest is extremely hard. People should not be pretending they can do it at high accuracy. They cannot. On the flip side, predicting someone needs a knee replacement is actually much easier. The factors behind it, and the timeline, are much more controllable."

Vishwanath has some other advice for health organizations looking to make hay with predictive analytics and population health management.

"Population health the way it's done today is wrong for two reasons," he says. First, too many organizations "study the entire population" when in reality it's usually only "the 3 percent, the tail, that really drives everything."

That other 97 percent? "Even if you do a bad job modeling them, it doesn't matter," he says. But that critical 3 percent, "you need to do a damn good job modeling them. You don't need population health. You need outlier health.

"You need to understand who those people are and do so as precisely as you can," says Vishwanath. "Know what trajectory they're on and what's going to happen to them, because that's what's going to bend your curve. If you try to model everybody, you'll do an average job on everybody."

The second fact too few folks are thinking about is the reality that "your population" -- even when honed down to those few outliers -- "is only about half of the equation," he says. "The other half is your physicians and your ancillary services.

"If you want to predict what's going to happen, that prediction is very physician- and facility-dependent," he adds. "If you think you're going to understand a population's trajectory with a predictive algorithm, you're missing some of the most important variables in the prediction. People say ZIP Code is very important. I agree. People say race is very important. I agree. Age is very important. These are all very important variables."

But to be blind to who the treating physician is means "you'll never be able to predict very well what happens," says Vishwanath. "Because at some point you're beholden to the physician's route of car: How they do the surgery, how they judge appropriateness. That portion, if you hide it from yourself, your predictions will be poor. The next generation will be looking at the entire system, the coupling between the physician and the patient."

Sriram Vishwanath presents on Thursday, Nov. 5, at the Healthcare IT News Big Data and Healthcare Analytics Forum. Register here.