Web Exclusive

‘Smart Pilots’: Digital Health Tools Transform Health Care
By Joseph Doyle, PhD, and Sarah Reimer, MD

A coming wave of digital health tools has the potential to transform how and where health care is provided. Using information from a patient’s medical record—including lab results, provider notes, and images such as CT scans—along with genomic data, prior insurance claims, and environmental information, machine learning algorithms can substantially improve diagnostic testing. They can also support decision-making tools for providers to improve guideline adherence.

The tools’ success is not a given, however. This new technology must first gain the trust of patients, providers, and payers. In addition, the tools must not prompt alert fatigue. If providers are flooded with warnings and advice, they may become desensitized and tune out the information.

This coming wave provides a golden opportunity to overcome both of these hurdles by establishing smart piloting of the new tools. By systematically introducing these digital health tools, we learn what works and what doesn’t. Randomized trials do not need to be limited to pharmaceuticals and medical devices—they can inform health care delivery designs as well.

A ‘Pilot’ Program
Recently, we worked with colleagues Sarah Abraham, PhD, Laura Feeney, and Amy Finkelstein, PhD, at the Jameel Poverty Action Lab, a research institute at MIT that supports randomized evaluations of this kind, to conduct a rollout of a new tool: clinical decision support (CDS) for high-cost imaging. This software provides information to doctors about whether a CT scan or MRI test they order for a given patient is appropriate. The information comes via a best practice alert, which is based on guidelines from the American College of Radiology.

Our study is timely. Starting next year, the Centers for Medicare & Medicaid Services has put in place a new regulation that requires a doctor’s imaging orders be accompanied by a CDS consultation. The regulation was put in place amid mounting concerns in the medical industry about the overuse of these powerful and expensive diagnostic imaging tests.

CT scans have become a routine medical test for Americans. In the United States alone, more than 70 million CT scans are performed every year. Some estimates suggest that as many as 30% of these imaging tests are unnecessary and that their overuse results not only in higher health care costs, but also increased exposure to radiation among patients.

Our trial, which took place at Aurora Health Care, the largest health care system in Wisconsin, involved 3,511 physicians, one-half of whom used the tool. The control group continued to order patient tests as they had prior to when the trial began.

Our findings suggest reason for cautious optimism. According to our study, CDS software leads to a 6% decrease of the images targeted by the CDS. CT scans—the most common generator of the warning due to its relatively high levels of radiation—were responsible for four-fifths of the overall reduction in targeted scans. Importantly, we found no evidence of alert fatigue reducing effectiveness over time.

We did not see a reduction in imaging, just a difference in the types of images that were ordered. This suggests that providers are reluctant to go without an image when they want one, but also that the software can help them choose the most appropriate one. Indeed, our study suggests that software alone can affect the appropriateness of ordering, but cost savings are less likely because the CDS often recommends an alternative high-cost image.

It’s true that a 6% decline in inappropriate ordering is only modest progress. But CDS is a relatively low-cost, low-touch intervention, and there are many ways to increase the technology’s efficacy. Further study of the most effective ways to employ the technology beyond simply showing the information about the guidelines—such as carrots and sticks for greater guideline adherence—remains an important area for future research.

Perhaps the most promising aspect of the study was the ability to randomize a new decision-support tool across physician practices as the tool was being launched. As we embark on a digital revolution in health care, smart pilots provide the rigorous comparisons necessary to make meaningful progress in improving health and lowering costs.

— Joseph Doyle, PhD, is the Erwin H. Schell Professor of Management and Applied Economics at the MIT Sloan School of Management and cochair of the health Sector at J-PAL North America.

— Sarah Reimer, MD, is the medical director of population health research at Aurora Health Care.