Clinical decision support systems (CDSS) aim to improve health care quality, safety and effectiveness by providing alerts to providers based on criteria (such as identifying drug interactions). However, a malfunctioning CDSS can actually reduce patient safety when physicians rely on these alerts.
According to “Analysis of clinical decision support system malfunctions: a case series and survey” by Adam Wright, et al, published March 28, 2016, “CDSS malfunctions are widespread and often persist for long periods. The failure of alerts to fire is particularly difficult to detect. A range of causes, including changes in codes and fields, software upgrades, inadvertent disabling or editing of rules, and malfunctions of external systems commonly contribute to CDSS malfunctions, and current approaches for preventing and detecting such malfunctions are inadequate.”
A survey that was part of the analysis found that 93% of Chief Medical Information Officers who responded had experienced at least one CDSS malfunction and two-thirds experienced at least an annual CDSS malfunction. Four such malfunctions were found within the CDSS system at Brigham and Women’s Hospital and were presented as case studies. We will examine one of these case studies within a Cause Map, or visual form of root cause analysis.
The first step in any root cause analysis method is to identify the problem. The CDSS malfunction in this case study involved a stopped alert for annual thyroid testing in patients prescribed amiodarone. When the issue was noticed and resolved in February 2013, it was determined that the alert had been stopped since November 2009, when the internal code for the drug amiodarone was changed.
An important step in describing the problem is to determine the organizational goals that were impacted. In this case, patient safety is impacted because of the potential for untreated thyroid issues and patient services are impacted because of the potential of missed testing.
The second step is to perform the analysis by developing the cause-and-effect relationships that led to the impacted goals. In this case, patient safety is impacted because of the potential for untreated thyroid issues. Patients may have untreated thyroid issues if they are taking amiodarone to treat arrhythmia. Amiodarone has a known side effect of thyroid issues. If staff is unaware of a patient’s thyroid issues, that patient won’t be treated. Staff would be unaware of thyroid issues in a patient if testing is not performed.
The goal of clinical decision support systems is to identify interventions based on patient needs – in this case, the hospital created an alert to suggest thyroid testing for patients who had been amiodarone and had not had a thyroid test in at least a year. Based on typical alert values from the years prior to 2009, the analysis determined that more than 9,000 alerts suggesting thyroid testing were missed.
Thyroid tests were missed because the CDSS did not identify the need for thyroid testing, and because physicians may rely on the CDSS to recommend a test like this one. The alert was originally set up to identify patients taking amiodarone (then code 40) with a start date at least 365 days ago, and no thyroid test values from within the last 365 days. In November 2009, the internal code for amiodarone changed to 7099, but the logic for the alert was not changed. (The reason for the code change is unclear.) As patient records were updated with the new code for amiodarone, the alert failed to identify them for thyroid testing.
The issue was identified during a demonstration of this particular feature of the CDSS and fixed the next day. While the details aren’t known, this issue identifies an ineffective change management program. When changes are made within systems, change management processes are necessary to ensure there are no unintended consequences. While updating the amiodarone code in the alert logic fixed this particular problem, a robust change management program is necessary to ensure that there are no other unintended consequences that could affect patient safety.
To view a visual root cause analysis of this example, please click on “Download PDF” above.