Demystifying the black box of data analytics

By John Criswell
11:00 AM
a:2:{s:5:"title";s:28:"John Criswell, CEO of Pulse8";s:3:"alt";s:0:"";}

Like never before, there is a data deluge in the healthcare delivery system. How healthcare organizations are bridging the gap between these large, disparate data sources and using them to impact the organization’s financial performance, improve member health, and alter member engagement will determine which organizations thrive in the future. The implementation of the Affordable Care Act and impending launch of the Health Insurance Marketplace brings additional pressure to quickly, or in real-time, process large amounts of data to (1) improve financial performance, (2) avoid unnecessary activities and (3) reduce costs. 

Having a powerful analytical solution is critical to accomplishing these goals. Whether organizations house their analytics internally or partner with outside firms to assist with developing and running the tools, the algorithms are often stored within a mystical black box that must be exposed.  The future of affordable health care is dependent on advancing the use of big data and analytics for bending the cost curve and improving quality of care. Exposing the black box provides insights and visibility that will help shape best practices, remove waste, and improve outcomes. 

Knowing what to ask

The black box requires organizations to trust that their analytics are executing each product and set of algorithms to the standards that were originally promised or designed.
This raises the question about the degree to which health plans, ACOs, and PCMHs now utilizing analytics can be sure that their black box is:

  • Protecting dollars in the net-zero Exchange environment
  • Improving targeting
  • Anticipating data gaps with newly insured beneficiaries
  • Appropriately deploying activities with limited financial resources
  • Managing effective activities with speed and accuracy in a condensed schedule
  • Improving confidence and precision in preparation for annual audits
  • Analyzing populations more objectively and independently

Even though powerful analytics are evolving within healthcare, numerous questions have arisen for the firms offering these tools:

  • What data sources are used?
  • How are the data enhanced?
  • How is the efficacy of the analytics determined?
  • Are new technologies and trends incorporated into the process?
  • How are the results shared?
  • How frequently are results available and to whom?

Answering each of these questions will allow a healthcare organization both insight and assurance that their analytics solutions are robust enough to meet the dynamic needs while staying relevant and innovative with their core products. Innovation and evolution are critical for any analytics solution because new data sources, conditions, methodologies, and technologies enter the healthcare landscape every day.

More data demands algorithm improvement

Data sources have undergone dramatic change in just the past five years. Indeed, 90 percent of the data in the world today was created in the last two years. In the past organizations could rarely rely on more than a claims dataset to input into its analytics solution, and the number of fields evaluated within claims was limited because most solutions focused on just a few parameters such as ICD-9 codes, Healthcare Common Procedure Coding System (HCPCS) codes, revenue codes, and places of service. Today, analytics solutions have a vastly larger set of inputs to consider including member-reported data, laboratory results, and vision data. In addition, modern solutions provide opportunities to create true clinical profiles, pattern recognition, event sequencing, and sentinel event benchmarking. And using the new datasets and combining them with “live” data such as EMRs/EHRs, ePrescribing data, and intervention results allow for much quicker and more accurate outcomes.

Each outcome from a single algorithm must be benchmarked against the expected result. For instance, if members with claims for home hospital beds are suspected for a recent stroke because of their immobility but the expected result yields a low confirmation rate of stroke diagnoses, then the algorithm must be modified to include additional factors such as a hospitalization or visit to a neurologist. This type of continuous algorithm improvement allows a firm to advance their products to best serve the needs of both their clients and the greater population. The origin of each algorithm, however, is just as important as the value of the outcome. Many organizations begin their algorithm development by using known clinical standards and associations. If a member is a smoker and has durable medical equipment claims for oxygen, they most likely have been diagnosed with COPD. But this type of algorithm development stagnates the innovation of analytics when unknown associations within a population aren’t considered. Organizations that develop analytics based on empirical evidence within large datasets are quicker to see new associations and implement those within their products.

Developing analytics based on empirical evidence also has numerous pitfalls that a healthcare organization should watch for. Identification of a disease or gap in care can often be the result of a mere coincidence in the data or from a mis-coded claim within the dataset. For instance, a member may have seen a cardiologist because of chest pain and that cardiologist may have incorrectly coded the member as having a heart attack in the recent past. Although coding standards do not allow for submitting rule-out diagnosis codes, these types of errors are still prevalent within the industry. A robust analytics product should evaluate each set of data to determine if a procedure, diagnosis, or even provider type exists as an outlier for that member. Such pieces of data should be excluded or weighted appropriately within the product unless additional data to support it becomes available.

The technology and computing power available today also allow analytics solutions to reach beyond algorithms that utilize simple association rules. Building clinical profiles of each member by benchmarking that member’s behaviors and clinical history against millions of other members, for instance, can produce deeply powerful analytics. Further, it is possible to establish pattern recognition and event sequencing to identify members that may have a worsening condition or an impending medical event that could be avoided through a deployment of interventions such as education, tests, or notification to their Primary Care Physician.
There are also great opportunities to utilize modern technologies. 

Mobile health devices are allowing organizations to close the data gap on newly insured members with little to no clinical history. Incorporating data from mobile applications can help to improve the confidence levels for suspected risk adjustment and care gaps while alerting members about how to better manage their health conditions. “Patients who were more knowledgeable, skilled, and confident about managing their day-to-day health had costs that were 8 percent lower in the base year and 21 percent lower in the next year,” according to the authors of a study published in Health Affairs.

Finally, any analytics solution should include business intelligence tools to demonstrate the performance of algorithms on your data over time. Ensuring a tool has the capability to display geo-mapping, benchmarks within the healthcare industry, financial impact, member profiles with individual risk factors, and expected realization rates are just a few elements healthcare organizations can review to evaluate their analytics solution or potential partners. Taking that visibility into the analytics solution and then connecting it to real-time data will reduce the delay between interventions and reporting while improving confidence in program results and increasing program performance.

Analytics market expanding

The field of healthcare analytics is exploding with new, as well as established firms, competing to capture business as many healthcare organizations pursue their own analytics solutions. Whichever solution an organization is considering, pry open the black box lid and uncover the performance of your algorithms, associations, and business rules.

Capture critical data sources such as member-reported data, and close the data loop to seek out the most efficient gap closure technique. Drive towards real-time visualization and reporting of your results so you can achieve the deepest insight into your population and its behaviors.

The performance of programs deployed for revenue optimization and the success with risk adjustment, quality, and cost containment on the exchanges will all depend on knowing what is inside the black box.

John Criswell is the founder and CEO of Pulse8, a healthcare technology and analytics vendor that focuses on business intelligence, commercial health insurance exchanges, and risk adjustment to close gaps in care. To read the Government Health IT interview with Criswell, see Q&A: On health data 'we can't dream big enough'.