Disaster Recovery — Lessons From Tulane’s Response to Katrina
By Alice Shepherd
For The Record
Vol. 21 No. 3 P. 10
Nowhere is speedy disaster recovery more vital than in a medical facility. For the many HIM departments across the country whose electronic health records are still incomplete, processes must be in place to protect and restore both paper and electronic information. The lessons learned by the personnel at Tulane Medical Center in New Orleans during and after Hurricane Katrina may be valuable to other facilities in their disaster preparedness efforts.
Keep Paper Records Out of Harm’s Way
The staff at Tulane Medical Center consider themselves fortunate that the majority of the facility’s paper records were stored in the second-floor HIM department, which was spared from the flood waters that accompanied the August 2005 hurricane. However, some records in other locations were damaged. Shelia Gorden, RHIT, HIM director and chief privacy officer, explains that “prior to Hurricane Katrina, most of Tulane’s clinics were located within our downtown hospital building, and their records were centralized in the HIM department. One exception was the cancer center, which was on the first floor of an adjacent building. Another exception was the sequestered records, which were temporarily housed in the medical affairs department on the hospital’s first floor.”
Fortunately, none of the first-floor records were completely destroyed because staff had the foresight to remove files from the bottom two shelves. However, because the flooded hospital was closed for several months, by the time HIM staff members were able to return, mold was growing on the walls and the humidity had damaged the records on the first floor. As a result, they needed to be sent out for restoration. “In areas prone to flooding, make sure that if your medical records are located at ground level or in a basement, you have a quick and easy means to relocate them to a higher level,” Gorden recommends.
When Hurricane Katrina struck, it was the first time that the Tulane HIM staff was declared nonessential personnel and asked to evacuate. This had two unfortunate consequences. One was that as patients came to the hospital, retrieving records became a challenge. In some cases, original records were sent with patients who were being evacuated. “Thankfully, our CIO [chief information officer], Sue Rachuig, was on site and knew that a log had to be kept of where the files where going,” says Gorden. “Just recently, we received our last record back from a hospital out of state.” Since then, Tulane’s IT department has created an external discharge transfer form that summarizes basic information and can be printed out to send with patients who are being transported to other locations.
The other consequence of HIM’s absence was that Tulane’s personnel couldn’t gather records from the clinics and hospital units to return them to the HIM department. “A couple of months after Katrina, while the building was being remodeled, myself, my assistant director, and another manager donned hazmat gear and went through all seven floors of the downtown building looking for records. We also searched the De Paul campus’ four buildings several times. It was a massive effort. Even today, people are still finding records in various places,” Gorden says.
As areas of the hospital campuses were being remodeled or closed, Gorden and her staff had to dig through trash bags in their search for patient information. “Make sure you get into the building before the construction workers throw everything in trash bags,” she recommends.
Consider Decentralizing Paper Records
Because downtown New Orleans was inaccessible for several months after Katrina, many of Tulane’s clinics began operating in various locations around the city. “As the downtown facility became accessible, we would enter the building, retrieve records, copy them, and fax them to clinics as needed,” says Assistant HIM Director Pamela Perry, RHIT.
Today, the clinics have their own HIM departments. “We decided to implement something we had already considered before Katrina—decentralizing the clinic records,” says Gorden. “HIM worked closely with clinic administrators to establish mini HIM departments, develop policies and procedures, and train staff, nurses, and physicians in the documentation and handling of records,” says Perry. “It was a new process for physicians, who were used to HIM delivering records to them and picking them up again.”
Now, HIM monitors compliance and serves as a consultant. “For example, one policy states that any clinic that is located on a ground floor has to move all paper records to a higher level if a storm is approaching,” says Perry.
The Value of an EMR
“In a disaster recovery setting, the ideal situation is to have electronic records that can follow the patients,” says Robert Lynch, MD, president and CEO of Tulane Medical Center. “Several New Orleans hospitals lost almost their entire medical records departments. Patients were evacuated to other areas of the country, and some who were being treated for cancer neither knew what treatment they were receiving nor what type of cancer they had.”
HCA, which is a joint owner of Tulane Medical Center, is implementing its electronic medical record (EMR) in phases, with the outpatient record scheduled to be completed by the end of this year and the inpatient record set for completion in 2011 or 2012. “The plan is to layer current electronic capabilities on top of the Meditech backbone and also feed data into a national data warehouse, which will permit the sharing of information across the entire HCA system,” says Lynch.
Prior to Katrina, most of Tulane’s diagnostic records (lab, radiology, cardiology) were electronic, along with some nursing documentation and a few dictated physicians’ notes. “We are currently training physicians on how to view documentation electronically and electronically sign their reports,” says Gorden. “They now understand the benefit of electronic records that are accessible from anywhere, even when paper records are in a building guarded by the National Guard.”
HCA already has a shared services model for IT and billing and is conducting pilot programs to centralize EMR operations. “We’re preparing to implement a totally electronic clinic record,” says Perry. “Physicians are going to document directly into the system. Nothing will be scanned in except records received from non-HCA facilities. We will be implementing an electronic physician order entry program and have begun to set up some of our coders to code off site.”
“Our data center is in Fort Worth, Tex., and we never lost any electronic clinical data,” says Rachuig. “As long as we had power during the storm, our users were able to access the electronic records. Once we lost power due to generator failure, they had to work with paper charts. However, after we evacuated, the physicians were able to dial into the Meditech system and access patient data remotely. Even though they were in other parts of the country, they were still caring for patients who had contacted them. Our HCA sister facilities also had access to the Meditech data they needed to care for the patients they had received.”
HCA has a number of “hot sites” (backup sites) for its regional data centers that are tested several times per year. If a data center goes down, the backup site takes over, reconfiguring the networks and getting all data back up in less than 24 hours. “Our electronic record is continually backed up to another server,” says Perry, “so if this system goes down or if we’re unable to access it from here, it’s just a matter of time before we are switched to the backup server.”
Prior to Katrina, Tulane’s HIM department was a 24/7 operation, with approximately 100 full-time staff working in three shifts. They handled all records for the downtown hospital’s clinics (then located on site), as well as the De Paul campus (which did not reopen after Katrina) and Lakeside Hospital in Metairie.
Immediately after the storm, only Gorden, Perry, and one other manager returned to work. Even several months later, Gorden still had only about one dozen staff members. “Clinics were opening, but employees had been scattered outside the city and out of state,” Gorden says. “When all the New Orleans hospitals began reopening, there was not enough staff to go around. Decentralizing the records was our only option and fortunately, it was something the physicians had wanted anyway.” Today, about 40 staff members handle the records of Tulane’s downtown campus and Lakeside Hospital and act as consultants for the decentralized clinics. The department operates only during standard business hours.
“We learned how to do things more efficiently,” says Gorden. “For example, we gave up part of our space for the physicians’ lounge. The staff physicians can use a swipe card to access the incomplete records room any time to dictate reports or sign off other deficiencies. Because we’re no longer there 24/7, this makes it much easier for them to complete their records while the department remains secure.”
Establish Means of Communication
Prior to Katrina, Tulane had an emergency contact list for each department, but it was limited to phone numbers. “Staff members were scattered outside the city and out of state, and we could not communicate because cell phone towers were not working,” says Gorden. “After a while, people learned to use text messaging and were able to learn each other’s whereabouts and e-mail addresses.”
“Now, our emergency contact list includes e-mail addresses, alternate e-mail addresses, alternate phone numbers, alternate addresses, and out-of-town addresses with names of next of kin where staff members might be staying in an emergency,” says Perry. HCA’s disaster recovery Web site also serves as a way to communicate information to staff. In addition, there are dedicated satellite uplinks, voice over Internet protocol telephone service, FM radio networks, satellite phones, and cell phones with area codes from outside the area to ensure uninterrupted data and voice communication.
Consider Posttraumatic Stress
When staff members return after a disaster, remember to be compassionate, caring, and flexible, Gorden says. “Work has to get done, but don’t forget about people’s emotional needs. Everyone needs to tell their story and share their experience,” she notes. “Some of us lost our homes, and all of us have relatives who lost everything. People’s families are still scattered. One of my managers is still driving back and forth from Baton Rouge, two hours each way.”
“People came back, but they weren’t emotionally here,” says Perry. “I lost my mother, and many other employees lost family members. Many elderly people died shortly after Katrina because the trauma they went through aggravated their medical conditions. Those are the statistics that are not counted.”
Tulane’s downtown hospital is currently undergoing major remodeling. “We’re coming back better and smarter because of new processes and investments,” says Gorden. “Excellent physicians, nurses, and other staff are joining us because of Tulane’s reputation, as well as a desire to help rebuild our area. We have cutting-edge cardiology and transplant programs, the cancer center, and other excellent programs.”
HIM and the hospital are well prepared for future disasters. “HCA hospitals in hurricane-prone areas have hardened their physical plants,” says Lynch. “For example, the first three floors are watertight so that generators and environmental systems are protected. Fuel is available to sustain the hospital and information systems for at least a week. Dedicated air and ground transportation assets will permit evacuation of patients before and after storms.”
As nature would have it, many of these new systems were tested recently during Hurricane Gustav. By all accounts, essential HIM personnel remained on site and everything went smoothly.
— Alice Shepherd is a southern California-based business-to-business journalist specializing in healthcare topics.
How to Choose Disaster Recovery Technology
According to Bill Lang, CBCP, CBCV, MBCI, business continuity program manager with Virtual Care Provider, Inc, healthcare organizations are not ready to interview vendors and compare systems unless they have first taken the following two steps:
• Understand your business operations and disaster recovery needs. Conduct a business impact analysis that prioritizes business functions and identifies the order in which they need to be brought up after a disaster.
• Conduct a threat assessment. Work with local emergency management to identify the site’s vulnerabilities (not just weather related but also issues such as proximity to a chemical plant and resiliency of local telecommunications).
Vendors will need the results of these analyses to recommend the most effective systems. “If a vendor doesn’t ask for this information, it probably means they don’t know what they’re doing,” says Lang. The following is his advice for proceeding with technology selection:
• Know the difference between four types of vendors. Some are basically landlords that take your equipment and people and just give them an alternate place to work. Others provide equipment, such as telephones, computers, and network connections. A third type offers mobile computing options (ie, semitrailer trucks with computing power, which can pull up to any location). The fourth type are hybrids that offer combinations of these three options.
• Be aware of cost structures. For example, most traditional vendors prefer clients who have an increasing need for their services (100 servers this year, 200 the next, and so on). They usually don’t accommodate a business that reduces its needs. For example, popular server virtualization efforts to reduce server count may not reduce server costs for disaster recovery.
• Keep both data and applications available at all times. Because people’s lives depend on the availability of medical records, hospitals need always-up, real-time replication. Electronic records create a total dependence on technology, making it critical for an infrastructure to be in place to keep both data and applications always available. High-availability technology fulfills this need. Several different types of high-availability systems are available and should be carefully researched.
• Keep abreast of new technologies. For example, vendors offering “cloud computing” or “utility computing” sell computing power on demand, which may come in handy for disaster recovery and business continuity. Make sure whoever negotiates with vendors understands the jargon and the limitations.
• Carefully read the contract. For example, if an automatic renewal provision is included, it may be difficult or impossible to switch vendors without penalties that are often disguised as termination fees. You can’t wait out the term of the contract with automatic renewal.
• Consider mutual-aid agreements with noncompetitive partners outside each other’s disaster zones or vendors who can ship equipment at a moment’s notice for systems with moderate to long recovery times. For example, if you have servers that don’t need to be recovered for four days, you can have those servers quickly shipped rather than pay a monthly fee to have them at your disaster recovery site. Make sure your vendor will allow you to ship your own equipment to its site.
Lang says that HIM disaster preparedness should be part of a comprehensive business continuity management framework, such as that developed by the Disaster Recovery Institute International. It includes planning for IT recovery, evacuation, emergency response, crisis management, and business resumption. Particularly important to hospitals is personal preparedness and awareness to ensure staff members can feel confident that their loved ones are safe while caring for others.