September 2019

The Emergence of Ambient Clinical Technology
By Susan Chapman
For The Record
Vol. 31 No. 8 P. 22

Set to debut next year, Nuance’s potentially groundbreaking documentation tool may reshape patient encounters. In this report, learn about its attributes and the hurdles it faces.

Nuance’s most recent product offering, Ambient Clinical Intelligence (ACI), is designed to securely listen to clinician-patient conversations and automatically document patient care, using machine learning and artificial intelligence (AI). ACI will build on the company’s existing system, Dragon Medical One, a cloud-based speech platform for clinical documentation, and offer assisted workflows and task and knowledge automation using specialized ambient sensing hardware. The technology will first be introduced to orthopedic physicians in early 2020 and roll out to other medical specialties over the next few years.

“Our goal in creating ACI is to enable health care providers to shift their focus away from their computer screens to their patients while still creating accurate documentation and smooth workflows,” explains Kenneth Harper, Nuance’s vice president and general manager of health care virtual assistants. “To accomplish this, we use a significant number of technologies to handle multiparty conversations in a passive way.”

Among the technologies that work in concert to create the ACI system are voice biometrics, which signals to the device when it’s time to begin recording, a process that starts when a clinician enters the room and makes an introduction or states that it’s time to start.

Speech signal enhancement is employed to “put a beam” on each person in the clinic room. For instance, there is a beam to the device that comes from a physician and one that comes from a patient. If there is a nurse, or, in the case of pediatrics, a parent, in the room, there can be a beam coming from that individual, too.

The technology utilizes multiple microphones and machine vision to record and track each unique person’s speech separately and suppress all other noises, enabling the conversation to be transcribed. The enhanced audio streams to the cloud, which takes the eight- or nine-minute, free-form conversation, transcribes it, and determines who is talking. AI technology then creates a clinical document.

Of the ability to distinguish among parties in a treatment setting, Harper says, “We’ve brought AI into everything we do, and we’ve been working on this for several decades. Therefore, we have become more efficient at handling multiparty conversations. ACI coordinates such a conversation in a passive way, listening to a free-form conversation and creating a document from it. You could be talking about your weekend and then go into talking about your symptoms, and the conversation, like all conversations, is interspersed with different details, some pertinent and some not. So, it’s a complex problem that we’ve set out to solve—how can we extract the salient points for the record?”

Harper explains there are other technologies at work to extract intelligence from those conversations in the cloud. “Not only are we transcribing what is being discussed, but we are also summarizing into a clinical note and identifying the discrete facts and plans related to that particular encounter. Summarizing entails taking a long free-form conversation around a chief complaint, examination, assessment, and treatment plan and distilling that down into a succinct and medically relevant clinical note automatically, akin to how a physician would have typed or dictated that same information manually,” he says.

While the technology may appear similar to home devices that understand commands and perform simple tasks, Harper emphasizes that ACI is more sophisticated in what it is able to achieve.

“Home devices focus on one-direction skills. The person in the home needs a task performed and commands the device to complete it, which presumably it does. In addition to supporting explicit physician commands for tasks such as finding patient information out of the EHR, the much bigger problem we want to solve is having a fully passive experience, where everything is done automatically behind the scenes based on the encounter conversation. ACI is much more about actually discerning and understanding conversations,” he says.

Nick van Terheyden, MD, CEO of Incremental Healthcare, believes the time has come for ACI. “I think we’ve been waiting for this type of technology for a considerable amount of time. It’s an inflection point and a redress of balance. EHR has been a time sink. Voice recognition has been a supporting actor. It didn’t solve the problem. Clinicians have to pay attention to laptops, screens, or telephones. The concept of a listening tool, an agent that sits in the room with a physician and patient, that listens to the conversation and produces documentation, is compelling. You don’t go and see your doctor to watch his back while he looks at a screen and types what you’re telling him. ACI allows the physician to look at, and be with, the patient,” he says.

Challenges
Experts believe, though promising, the technology still has hurdles to overcome.

“The technology is ready but not in all instances,” van Terheyden says. “When you look at the concepts in an ideal setting, a clinic setting, a single room with good audio characteristics, not noisy, just a room with a single physician and patient, it works well. But it doesn’t really work in a busy ER, a mobile setup. You can’t just wheel it in. ACI has been fine tuned for specific circumstances, around orthopedics. If you were to move the device to the renal clinic, for example, it would need training. ACI needs to be agile and highly accurate to deliver what it promises. The last thing you want it to do is to create a document that is incorrect. You must be thoughtful and focused on where you apply the technology, from a specialty or circumstance standpoint.”

Like many transformative innovations in health care, Harper agrees there are challenges to overcome, but he views ACI as more of an evolution, not a revolution, from a technological standpoint and notes the company is already on a path to overcoming many of the potential hurdles.

“The speech-signal enhancement technology tries to figure out where someone is through an array of microphones and advanced signal processing,” he says. “By having speech and audio input being picked up by multiple microphones, the software allows you to infer where someone is in relationship to the device, such that algorithmically you can home in on that relevant conversation and suppress everything else. Today we are already seeing the room doesn’t need to be completely quiet for ACI to work well.”

Harper offers a common scenario to demonstrate ACI’s effectiveness. “In a hospital, it’s not unusual for a TV to be on in the room where a doctor and patient are talking. In those circumstances, software can infer where the doctor is and ignore everything else. If you were to listen to a before-and-after recording, you would hear the room one way, and then you would hear only the doctor and patient talking in the ACI audio stream,” he says. “That’s because, through advanced signal enhancement, we are able to get high levels of accuracy in multiparty conversations. We don’t want to dilute the ultimate quality of the system.”

Initially, to provide a quality check and produce training data for the AI models, Harper says the ACI output will be reviewed and annotated by a person with medical expertise. Once the system has proven its accuracy, ACI documentation will go directly to the physician for review and ultimate sign-off.

Karen Youmans, MPA, RHIA, CCS, president of YES HIM Consulting, believes that because the device is computer driven, there are potential issues that could arise.

“In the English language, there are so many variations and different accents and dialects. AI is getting better in administering these variations, but currently our auditing team still sees the need for additional tweaking and editing,” she says. “Health care providers continue to need to be editors to provide feedback in order to keep upgrading the quality of information managed by NLP [natural language processing] and AI. And the more feedback they get, the better that technology will become.”

However, Harper believes today’s dictation solutions, which leverage AI and enormous data sets of various dialects and accents, have already overcome the issue of diverse speech patterns. He says training is not required because ACI leverages many of the same foundational technologies, with new advances in conversational handling being added.

“Current systems, given the changes in machine learning, have helped us produce devices that are highly accurate right out of the box,” Harper says. “We have spent a great deal of time addressing how best to collect the audio from the person who is talking and where that voice is coming from. There is a lot of new technology, but, at the end of the day, the foundational technology pieces are based on those models that we’ve collected over the past couple of decades, and the speech accuracy is quite high.”

He adds that although ACI is unable to offer coding capabilities when it is first incorporated into the clinical setting, there will be a coding element that ACI will provide over time. “ACI will play a role in coding. There will be certain facts, and we want to get the relevant facts for the EHR and extract the relevant codes. How much of the document gets coded automatically over time, that will be increasing as we go along,” Harper says.

Youmans says that the coding aspect of ACI could pose a potential issue. “With AI and computer-assisted coding [CAC] programs, the health care providers do not necessarily provide feedback for the vendor. In fact, with any CAC system, the system will autoselect some codes, but the coder has to be the editor of those codes.

“The system doesn’t know the ins and outs of all the official coding guidelines. It may know the codes for signs and symptoms, but the coders need to know if those signs and symptoms are integral to the diagnosis,” she explains. “The view of that particular technology was more optimistic than it turned out to be. The rules and guidelines still need to be interpreted, and the coders need to edit and be more specific in terms of the codes. Time will tell regarding ACI’s efficiency in this area.”

While the system’s functionality and potential coding issues are of concern, van Terheyden notes that the privacy element seems to have been addressed. He points out that ACI’s presence is unlike that of medical scribes, who are agile documenters for busy clinicians but who can feel like interlopers in sensitive situations in which patients may want to speak with their physician alone.

“Technology can improve that type of situation, but sometimes not,” van Terheyden says. “Patients will wonder, ‘Is this thing recording me?’ even though it’s not actually ‘seeing’ the patient. Within the device, there appears to be ‘eyes’ and cameras. But, instead of putting actual cameras that would provide clear identification of an individual into the ACI device, Nuance uses sensors that ‘sense’ the people in the room. It doesn’t show a face, but rather a construct. It creates context; it gives the context that a scribe would have without the privacy issues because it doesn’t show the person per se. And while patients may have privacy concerns, in actuality, Nuance has done a good job to address them.”

Harper concurs that Nuance has anticipated privacy issues and has other safeguards—along with the sensor technology—in place to solve them. “With ACI, there must be user consent and full disclosure. ACI is there to help the physician with the treatment plan, but patients have the right to say no, and then the device can’t record,” he explains. “Full transparency must be there as we bring this device to market. There are already virtual scribing solutions today. A doctor enters a room, for example, and she is recording the encounter on her phone. For this device, we have considered how to pause a recording if necessary. We need to have user control and programmatic controls in place so that the device stops recording automatically, and we do.”

Another potential issue centers around how the documentation of such encounters is produced, which creates uncertainty, the potential for misunderstanding within the record, and legal challenges should a problem arise.

“If the technology creates it, then it needs to be acknowledged and reviewed by the doctor and even the patient,” van Terheyden says. “The doctor and patient can pick out errors in the documentation. But once the doctor reviews and signs off on the record, you have to discard the original recording that the ACI device created in order to avoid potential conflict downstream.”

“There are a lot of questions amongst the health care community about whether or not these recordings should be shared back with the patients at all,” Harper says. “There are two camps. In general, one camp is of the mind to share the recordings with the patients. A patient could review it at home and maybe share it with a family member. If there are very specific treatment plans that are being discussed, having a copy could be incredibly helpful to improve outcomes. The other camp, though, maintains that under no circumstance should data be shared with patients because it increases the chance of liability in malpractice. In the latter case, there would be no conflict with the record produced because the patient simply would not have access.”

In today’s health care environment in which data breaches are a major concern, the possibility that ACI would become another victim is a legitimate concern. Harper acknowledges such a possibility but believes the necessary precautions have been taken.

“It would be disingenuous for me to say that ACI would never be used for a nefarious purpose if we were to completely open it up to the outside world. However, we have our own vision about what’s going to happen with this device now that we are bringing it to bear in the marketplace,” he says. “There is no reason, in the future, this device can’t be a patient-aiding device. For instance, we envision this device engaging with a patient in a way that helps do the preappointment paperwork, eliminating much of the time spent in physicians’ waiting rooms, wading through clipboards full of paperwork.

“There are so many possibilities to capture real-time information. More and more services can come from this. And there has to be a regulated process that ensures the right services come from it. Think about Apple and its app store; they have a carefully controlled system regarding who can and cannot publish an app. That level of control is the same for this technology. We must ensure that tight safeguards are in place, and we have.”

 — Susan Chapman is a Los Angeles–based freelance writer.