February 2014

Assessment Tool Promotes EHR Usability
By Selena Chavis
For The Record
Vol. 26 No. 2 P. 26

The EHR movement has reached a crossroads over the past year, and usability has become a key focal point of forward momentum. A vendor market that grew to $20 billion in 2012 is facing an uncertain future as initial grant money dries up and numerous reports suggest that a sizeable percentage of companies will not have enough staying power for the long term.

Specifically, one report from Black Book Rankings named 2013 the “year of the great EHR switch,” with providers frustrated over their current system’s limitations to meet the future expectations of an evolving, performance-based regulatory landscape.

By all appearances, meaningful use is achieving its initial goal of advancing EHR uptake, but experts warn that the usability of these critical systems will be important to advancing current industry goals.

“While EHRs have a lot of potential to bring positive impact, if they are not designed well, they will hinder progress,” says Muhammad Walji, PhD, associate director of the National Center for Cognitive Informatics and Decision Making in Healthcare (NCCD), a consortium of institutions led by the University of Texas Health Science Center at Houston (UT Health). He also is associate dean for technology services and informatics and an associate professor in the School of Dentistry for UT Health.

As part of an effort funded through the Office of the National Coordinator for Health Information Technology’s (ONC) Strategic Health IT Advanced Research Projects to address EHR adoption challenges and long-term progress, Walji and a team of NCCD experts were assigned to develop tools to effectively test EHR usability. The effort has produced the TURF (task, user, representation, and function) framework for usability and the TURF suite of software tools, or what the NCCD refers to as an all-in-one “usability toolkit.”

“It’s a tool but also a guiding philosophy about how you think about design,” Walji explains, pointing out that the program aims to lay a foundation for future EHR design.

TURF Overview
According to Walji, the basis for the TURF philosophy stems from cognitive science, beginning with a preliminary understanding of the work that occurs in a particular domain and user characteristics. The task component represents a detailed evaluation of the physical and mental tasks that go into using an EHR, while user represents an understanding of how humans interact with the software and hardware. “Once you know about functionality needs, then you can determine the best way to approach interfaces and order workflows,” he says.

Representation, which characterizes an EHR’s user interface, is essentially where the “rubber meets the road” in terms of usability, according to Walji. Finally, the function component involves analyzing the functions within the work domain—the work that will be performed in tandem by the user and the system.

By adopting this framework, Walji says providers can better understand why two EHRs can represent the same work domain in different ways. TURF also can determine the best representation for each of the identified areas.

What TURF Offers
For the purposes of defining its approach to evaluating EHR usability, the NCCD narrows the focus to three attributes: useful, usable, and satisfying. Essentially, an EHR is useful if it gets the job done.

That’s the easy part, according to Walji. On a more complex level, usability measures how difficult it is to become proficient using the system, how efficiently the system completes tasks, and the system’s error tolerance. Satisfaction is, of course, about the overall user experience.

According to Walji, the TURF software tool delivers several usability assessments, each supporting two techniques: expert review and user testing.

The heuristic evaluation, or expert review, collects data to compare against usability standards. Reviewers can mark screens, denote and rank levels of concern, and compare results with other reviewers. Walji notes that this evaluation method traditionally has been completed with pen and paper, making it a laborious, time-consuming endeavor. The TURF tool enables a more efficient process by generating reports to make statistical comparisons against usability standards and benchmarks based on the National Institute of Standards and Technology’s EHR usability protocol.

User testing allows live assessments of the user experience by guiding clinicians through a customized set of tasks. During user encounters, the software captures details of the experience through audio, screen capture, and keystroke events.

Tests can be conducted for broad application as well as to determine usability for a specific function, such as ePrescribing. Evaluation can occur at both the formative and summative development stages, meeting a requirement laid out for certification under stage 2 of meaningful use. “Before physicians can get meaningful use dollars, they have to buy a certified product,” Walji says, pointing to the rule requiring vendors to include evidence of user-centered design and user test results. “This is a tool that can facilitate those kinds of assessments,” he notes.

Specifically, the ONC has set certification standards for safety-enhanced design, making patient safety a focal point of EHR infrastructures. The certification requires a formal user-centered design process during development and summative testing that chronicles the amount of time necessary to complete a task, the number of errors that occur, and user satisfaction.

An extensive process, summative testing covers the following eight domains, each of which must be reported separately:

• computerized physician order entry;

• drug-drug/drug-allergy interaction checks;

• medication lists;

• medication allergy lists;

• clinical decision support;

• electronic medication administration record;

• ePrescribing; and

• clinical information reconciliation.

Walji points out that the testing tool does not solve all problems. Ultimately, it takes effort on the part of vendors to enhance usability accordingly. “It does put responsibility on those applying software,” he says.

Tutorials for the testing tools, which can be found on the TURF website (www.turfusability.com), offer comprehensive coverage of nearly 30 areas in the program. While the NCCD continues to add more resources for safety-enhanced design and EHR usability to its website, other design guidelines have been posted at https://sbmi.uth.edu/nccd/sed/briefs.

Looking Ahead
The NCCD hopes its work helps foster EHRs that provide patient-centered cognitive support, a goal that will be achieved when the technology is “designed and optimized to support problem solving and decision making that maximizes the chance of providing the highest quality of care for patients as measured by IOM’s [the Institute of Medicine’s] six dimensions of quality (safe, effective, timely, efficient, equitable, and patient-centered).”

Walji believes that adding the usability equation into EHR certification requirements is an important step toward realizing this vision. “I definitely want to commend the ONC for incorporating usability into stage 2 meaningful use,” he says, noting that usability was a hard sell when the incentive program was getting started three years ago. “We think that is a major step forward, and we hope that the momentum will continue into stage 3.”

— Selena Chavis is a Florida-based freelance journalist whose writing appears regularly in various trade and consumer publications covering everything from corporate and managerial topics to health care and travel.