May 2017

CDS Systems: Common Malfunctions, Practical Solutions
By Juliann Schaeffer
For The Record
Vol. 29 No. 5 P. 10

The power of clinical decision support software is undeniable, but users must be cognizant of its potential flaws.

With an overarching objective to aid decision making, clinical decision support (CDS) technology aims high—to improve health care quality, safety, and even efficiency. As more health care systems adopt CDS systems, it's essential to examine the technology's potential benefits as well as possible drawbacks.

Where can CDS systems fall short? More importantly, how can health care organizations be proactive in managing the issues, possibly preventing CDS pitfalls before they occur? Industry insiders weigh in with practical strategies and solutions to what they view as the most common problems associated with CDS use.

CDS Benefits Abound
CDS' top priority is to help clinicians make the right decision, at the right time, for their patients, says Natalie Pageler, MD, chief medical information officer at Stanford Children's Health and Lucile Packard Children's Hospital. "CDS systems augment clinicians' knowledge and decision-making ability by providing access to the right information at the right point within a clinician's workflow to help improve safety, quality, and efficiency," she says, noting that more advanced CDS technology offers even greater potential. "Advanced analytics is making clinical decision support tools more valuable by helping providers to process rapidly growing amounts of data from many different sources, ranging from gene sequencing data to patient-generated health data from home monitoring devices."

"Whether the focus is avoiding error through alerting mechanisms or presenting context-specific evidence-based guidelines when placing orders, the ultimate goal is to improve the quality of care our patients receive," says Erin Jospe, MD, chief medical officer at PatientKeeper.

"For example, a patient with COPD who is diagnosed by the physician can then be 'assisted' or supported by a CDS system by reminding the physician of the latest guidelines and/or reminding or proactively determining whether the patient has been on a smoking cessation program," says Geeta Nayyar, MD, MBA, chief health care and innovation officer for Femwell Group Health and host of TopLine MD TV.

According to Steve Oden, senior vice president of product operations at medCPU, CDS technology's most important function lies in its ability to transform critical patient data into actionable alerts. "Clinicians in all health care settings are increasingly flooded by information," he says, "so extracting the most relevant and time-sensitive data is essential in ensuring the best possible outcomes."

CDS technology is perfectly positioned to help health care organizations manage large amounts of information efficiently, says Mark Wolff, PhD, chief health analytics strategist at SAS, which specializes in advanced data analytics solutions. "These data are at the heart of delivering timely and appropriate care to patients, but the exponential explosion in digitized data has essentially crippled hospitals' abilities to effectively use them."

Wolff says CDS systems deliver value in two key ways: creating more efficient workflows and improving care via diagnosis, treatment, and other recommendations. "Efficient and accurate data workflows aid the entire hospital community by reducing the need for paper-based systems and increasing the speed and accuracy of information exchange," he explains.

"More 'intelligent' CDS systems are designed to aid the clinician not only in diagnosing a patient but also in making recommendations as to appropriate treatments," Wolff continues. "Such systems are either knowledge based, where specific clinical rules are encoded into a decision/recommendation engine, or they use computational techniques like machine learning and cognitive computing to process massive amounts of data to support clinical diagnostic and therapeutic decision making."

He notes that CDS technology can improve patient safety, quality of care delivery, and patient outcomes while at the same time reducing workload and costs.

Risks and Ramifications
Like any technology, CDS is not perfect. Although CDS systems have been invaluable at Stanford Children's Health, Pageler says unintended consequences such as alert fatigue and automation bias or complacency did occur. "Clinical decision support also has the potential to become outdated or malfunction without appropriate maintenance and review programs," she adds.

"One of the biggest risks is assuming all CDS systems are alike," Oden says. "One major challenge, highlighted in depth by a 2014 AHRQ [Agency for Healthcare Research and Quality] study, is implementing CDS tools across multiple platforms and accounting for differences in technology systems and infrastructure."

Wolff agrees, noting that interoperability and data quality concerns rank high among CDS risk factors. "If systems cannot talk to each other, and if we lack confidence in the accuracy and quality of the data, then there is great risk to the patient," he says.

While the health care industry has made huge inroads into becoming digitized, it still lags behind other industries—and that's a problem. "Much of this risk [of CDS] derives from the lack of sophistication and interoperability of clinical information technology infrastructure," Wolff says. "While most industries have adopted and developed highly efficient enterprise data management infrastructures, the health care industry has lagged in developing an equal level of IT sophistication and interoperability. The industry still struggles with the adoption of EHR systems that are more than just the digitized version of a paper workflow. This, to me, is an indication that it will likely be some time before the industry catches up."

Nayyar believes leaning too heavily on CDS technology can be dangerous. "CDS systems will never be able to replace the thought process and comprehensive analytical skills physicians use when assessing a patient," she says. "A one-size-fits-all regimen or system won't generally work or substitute for a physician's assessment. Support is terrific, but substitution or overreliance could have negative outcomes on patient safety and outcomes."

The ramifications of a CDS-generated error vary but they can be as impactful as any of its potential benefits. "Malfunctions can result in everything from the mundane annoyance to true harm," Jospe says. "As such, incredible vigilance is necessary for the maintenance and ongoing evolution of CDS systems. Any and every occurrence has possible significant patient harm associated with it, and even the least noteworthy consequences will still erode clinician faith in the reliability of the CDS tools as a whole."

The consequences of errors are painfully clear, says Wolff, who cautions that poor data quality can lead to substandard and erroneous decision making, which in turn can result in patient injury or death. Alert fatigue, the phenomenon in which clinicians believe warnings to be false-positives, can have disastrous consequences.

"Alert fatigue can be defined as sensory overload when clinicians are exposed to an excessive number of alarms from critical clinical devices used in the treatment of patients," Wolff says. "This overload can result in desensitization to alarms and missed alarms, which may lead to patient injury and death."

In recent years, The Joint Commission and the American Association of Critical-Care Nurses have identified alert fatigue as a major patient safety concern. "These organizations and more around the globe have realized through multiple research studies with compelling empirical evidence that alert fatigue is directly responsible for an ever-increasing number of patient injuries and deaths," Wolff says.

This raises the question of how clinicians—and patients, too—should view CDS. "Clinicians and patients must recognize CDS as one more tool to support the delivery of the highest quality of care and use it to augment their decision making but not to replace critical thinking," Pageler says.

Common Problems and Practical Solutions
In an effort to identify where CDS systems can go astray, Adam Wright, PhD, an associate professor of medicine at Harvard Medical School and Brigham and Women's Hospital, led an investigation into the matter at Brigham and Women's Hospital.

While the findings, published in the Journal of the American Medical Informatics Association, pinpointed faulty alerts and inaccurate recommendations as potential CDS shortcomings, perhaps the study's biggest takeaway was that 93% of chief medical information officers who responded said they had experienced at least one CDS malfunction. In addition, two-thirds said they encountered malfunctions at least once annually.

 "[The study] actually started from my own experience," Wright says. "I was giving a demo of our EHR and some clinical decision support that was supposed to fire didn't—everyone's fear when doing a demo. After we looked into it, we found that an underlying drug code for amiodarone had changed, but the rule wasn't updated."

The research team was surprised to learn that CDS malfunctions were actually pretty commonplace. "The problem had been described only a couple of times in past literature, and we assumed it would be rare because we have a substantial investment in knowledge management resources," Wright says. "We didn't expect to see so many issues. But clinical decision support systems malfunction fairly often and for a variety of reasons."

How can health care organizations tackle CDS malfunctions, particularly if they're difficult to detect? Experts suggest the following actions:

Practice regular monitoring and maintenance. Oden says it can be difficult to make generalized recommendations, especially considering CDS can apply to a wide array of technologies. However, he says often times malfunctions go undetected when CDS integrates with multiple technologies.

"CDS is often overlooked with respect to change management, validation, and operational monitoring practices in a health care setting," Oden says. "Finding a CDS tool that is backed up by the right technology infrastructure to allow for routine and easily repeatable CDS validation, as well as vendor support or vendor-provided tools that allow for operational CDS monitoring of alerts, is one of the best ways to prevent [common] types of [CDS] malfunctions."

"Much of the benefit of CDS tools stems from eliminating human error," Jospe says. "But I think we underestimate the effort of human maintenance of the CDS tools we have. The rules engines we build need constant reexamination, and organizations must plan for the downstream effects of seemingly unrelated changes, such as a drug dictionary change or a system upgrade as alluded to in [Wright's study].

"Quality health care delivery depends on the entire care team working together," she continues. "We need to remember that our computer workflows are entwined and interdependent. Communication between all the parts is key."

Invest in robust CDS planning and programs. Jospe believes smart organizations not only prioritize technology maintenance but also account for it up front, developing clear communication and planning processes at the start of a CDS implementation. "There needs to be a willingness to acknowledge how entwined seemingly separate workflows are and to recognize those stakeholders when making changes," she says. "It is not an easy undertaking, but I believe it builds stronger and more valuable tools as well as stronger health care organizations in the long run."

Pageler urges health care organizations to develop robust programs that thoroughly test, actively monitor, and regularly update CDS tools. "These programs should include active clinicians with knowledge of informatics to help design and appropriately test CDS tools," she says. "We [Stanford Children's Health] have a clinical informatics fellowship and a clinical informatics resident rotation where clinical residents and fellows are trained in principles of decision support design and the unintended consequences of CDS.

"We also have an active physician informaticist program, which includes six practicing physicians who are board certified in clinical informatics and actively participate in our clinical decision support committee. We've also created a novel nursing informatics council which is part of our nursing shared leadership that trains frontline nurses in key principles of clinical informatics and involves them in the design and testing of clinical decision support tools."

Partner with the right people. Sometimes effective monitoring comes down to forming effective interdepartmental relationships, Pageler says. "It is incredibly important for the information services departments to partner closely with the quality and patient safety departments to monitor and address issues with CDS," she says. "At Stanford Children's Health, we have robust reporting tools both specifically related to patient safety incidents and a more general information systems reporting tool. We also participate actively with our patient safety team and engage in all root cause analysis of any issues that may have involved information systems."

Choose the right vendor. "Avoiding problems with CDS starts with a health care organization partnering with a CDS solutions provider that is committed to supporting the organization as it manages change and monitors the operation of their HIT solution," Oden says.

While no single approach of checks and balances is perfect, he says open communication lines between HIT and clinical leaders as well as change management strategies can help avoid issues. "Clinical leaders and HIT leaders should have a process in place to ensure they're communicating on a regular basis with CDS end users so issues that are seen and easy to ignore … are able to be identified, investigated, and resolved," Oden says.

Start with a solid IT foundation. According to Wolff, the question isn't whether CDS is somehow deficient or unreliable. "Rather, the issue of malfunctions is typically more about the data—data management, data quality, master data management, quality control, and workflow management, and the integration and interoperability of hospital IT systems."

Effective data management is grounded in a sound IT infrastructure. "As a foundational first step, health care organizations must create the appropriate IT infrastructure to handle external and internal data flow appropriately," Wolff says. "The second and equally important step is to build upon that IT foundation an enterprise quality data management/data quality system that spans all tools and applications within and across the hospital system."

Couple awareness with an industrywide effort. Based on his research, Wright offers a three-step approach that he believes health care organizations can implement to address CDS concerns. "First, I hope organizations are aware of the potential of their CDS to malfunction—knowing is the first step. Second, I think organizations need to invest in better monitoring, as some CDS malfunctions result in substantial spikes or drops in the rate at which CDS fires. If you are looking for these, you can head off many problems," he says. "Third, I think health care organizations and EHR vendors need to work together to develop new tools and processes to make CDS more reliable."

In the end, Nayyar says health care professionals must view CDS technologies in the correct light. "They are meant as support tools and should be used as such," she says. "Problems will occur no matter what you do, as nothing is ever 100%, but ensuring good testing methods and communication pathways between IT and the clinical side of an organization is important. Together with a strong vendor partner that is aware of this issue as a priority can ensure a good process and maintenance going forward."

— Juliann Schaeffer is a freelance writer and editor based in Alburtis, Pennsylvania.