How University of Rochester uses AI to reduce risk of failed follow-up

The University of Rochester Medical Center uses analytics with natural language processing to support the organization’s Backstop tracking program at six of its hospitals. It has seen a 29% increase in the recommended examination completion rate.
By Bill Siwicki
02:18 PM
How University of Rochester uses AI to reduce risk of failed follow-up

The University of Rochester Medical Center needed a better way to ensure that its many patients with incidental radiology findings received their recommended follow-up care in a timely manner.


Failure to follow up happens for a number of reasons, including inconsistent communications during care transitions, not notifying patients of actionable test results, and inadequate systems for managing and tracking incidental findings.

“While a majority of incidental radiology findings are, thankfully, benign nodules, others turn out to be treatable forms of cancer,” said Dr. Ben Wandtke, medical staff president and chief of diagnostic imaging at UR Medicine Thompson Health and an associate professor in the department of imaging sciences at the University of Rochester Medical Center. 

“Unfortunately, due to gaps in communication, patients can fall through the cracks and not get into the proper cancer treatment right away," he said.

For example, the difference between a stage 1 cancer diagnosis and a stage 4 diagnosis can be a 50% or greater survival impact. Earlier diagnosis leads to improved patient care and better outcomes and can even result in a lower cost of care.


In 2015, the provider organization piloted a recommendation tracking system it calls “Backstop” at FF Thompson Hospital, a five-radiologist affiliated community hospital. The goal of the system was to serve as a safety net for patients for whom clinicians had identified a potential malignancy or aneurysm and offered an actionable recommendation.

“It was quickly a success,” Wandtke reported. “In the first 13 months, we tracked 589 recommendations, 86% of which were satisfactorily closed through the program, reducing the risk of delayed diagnosis by 74%. We subsequently expanded Backstop to include six hospitals and 75 radiologists.”

"Our tracking efforts directly lead to more than 1,000 new exams performed annually, often with higher-reimbursing imaging modalities such as CT and MRI."

Dr. Ben Wandtke, University of Rochester Medical Center

While the growth was exciting to staff, they soon realized that the manual process required to flag recommendations was a significant barrier to widespread adoption of the program. The Backstop program depended on the radiologist manually adding patient cases to a central database for tracking at the time of dictation.

“Our facilities conduct more than 800,000 diagnostic imaging exams annually, and our radiologists were too busy to consistently remember to add all of their recommendations to the database,” Wandtke explained. “We needed a way to automate this process to improve the effectiveness of our safety net.”

That's when the organization turned to artificial intelligence (AI) and natural language processing developer Nuance.


There are a number of vendors of natural language processing-powered analytics on the market today. They include Accenture, Cornerstone Solutions Group, Expert System, IBM, Lexalytics and SAS Institute.


Wandtke said that Backstop needed to take advantage of recent advances in natural language processing algorithms to help identify and track more of the recommendations coming out of the department.

“Based upon our pilot results, it was clear that we were capturing less than half of our recommendations, and we did not have the resources internally to develop a solution,” he said. “We partnered with Nuance to integrate mPower, their NLP-based clinical analytics solution, into our Backstop system. mPower contains radiologist-designed and -validated algorithms that automatically identify actionable recommendations from unstructured radiology report text.”

Without disruption to the provider organization’s radiologists’ workflow, the NLP-based clinical analytics provided another layer of protection for patients, he added. Additionally, the analytics is used to provide a report detailing the frequency and quality of recommendations generated to individual radiologists and leadership to improve the quality of recommendations radiologists include in their reports, he explained.

“Every radiology report from all University of Rochester hospitals and imaging centers is now analyzed using NLP algorithms daily,” Wandtke said. “The recommendations identified by mPower are compared to those entered by the radiologist. All recommendations not entered manually are automatically extracted into our database called Follow-up Manager, also developed collaboratively with Nuance.”

Program navigators now can review the NLP-extracted recommendations and with a few clicks populate the database with dozens of data points including patient identifiers, ordering and primary care providers with their contact information, and details of the incidental finding including recommendation due date and imaging modality.

The database provides a working list of overdue recommendations that need review, can generate templated letters and EHR notifications, and documents all communications with patients and healthcare providers.

“With the aid of these two medical informatics solutions, we have exceeded the pilot project results on a large scale,” Wandtke said. “To bring this back to the level of the individual patients we treat, I would like to share a real success story. We had an 80-year-old female patient present to the ED with shortness of breath. A chest X-ray revealed a worrisome nodule for which a CT chest follow-up was recommended.”

This recommendation was communicated to the ordering provider and documented in her report. The clinical analytics identified this patient and pulled the follow-up recommendation into the follow-up manager system for tracking via the Backstop program.

“After the second-level intervention in which the Backstop navigator contacted the PCP’s care manager, the patient received her follow-up CT three months after her ED visit,” he recalled. “She subsequently received a PET scan, which confirmed a stage 1 lung cancer diagnosis. Without Nuance’s solutions, this patient would likely have developed advanced stage cancer prior to diagnosis.”


The University of Rochester now tracks more than 500 recommendations per month (0.8% of diagnostic volume), and 91% of these recommendations are satisfactorily closed. The tracking system enabled by the follow-up manager system resulted in an 80% increase in staff’s ability to “close the loop” in patient care – reducing the risk of delayed diagnosis and ensuring timely follow-up on incidental findings, Wandtke reported.

Further, 55% of the recommendations tracked have been identified only by the NLP analytics.

“As a result, we’ve increased the recommended examination completion rate by 29% – from 55% to 71% – improving both patient outcomes and reimbursements,” he said. “Our tracking efforts directly lead to more than 1,000 new exams performed annually, often with higher-reimbursing imaging modalities such as CT and MRI.”

Beyond generating additional examinations, the program also improves patient care through earlier diagnoses and reduces medical-legal risk, he added.

“That said, it’s more than just tracking follow-up compliance,” he stated. “Programs like these reinforce the use of evidence-based guidelines and consistent application of high-quality care. They address the fragmentation of healthcare delivery and prioritize the patient and clinical outcomes.”


“Every patient deserves their best chance for a cure,” Wandtke said. “If there’s a technology out there that helps make this a reality, you absolutely should embrace it. The problem of delayed diagnosis related to actionable radiology findings is universal, but it does not have to be. Natural language processing-based analytics coupled with database technology and a human touch have nearly eliminated this problem at the University of Rochester.”

With healthcare moving toward a high-reliability model, provider organizations must develop systems of care that can identify small problems and fix them before they develop into crises, he added.

“I encourage other healthcare organizations,” he advised, “to seriously look at their gaps in communication, identify the barriers and work on a solution.”

Twitter: @SiwickiHealthIT
Email the writer:
Healthcare IT News is a HIMSS Media publication.

Want to get more stories like this one? Get daily news updates from Healthcare IT News.
Your subscription has been saved.
Something went wrong. Please try again.