Medical Error Prevention
and Patient Safety

Objectives: Upon completing this course the student will have an understanding of the significance of medical errors, the types of errors, reporting requirements and methods, costs of errors, ways to prevent and reduce errors and improve patient safety as well as current research into medical errors and patient safety.


An Epidemic of Errors

Where Errors Occur

Errors occur not only in hospitals but in other health care settings, such as physicians' offices, nursing homes, pharmacies, urgent care centers, and care delivered in the home. Unfortunately, very little data exist on the extent of the problem outside of hospitals. The IOM report indicated, however, that many errors are likely to occur outside the hospital. For example, in a recent investigation of pharmacists, the Massachusetts State Board of Registration in Pharmacy estimated that 2.4 million prescriptions are filled improperly each year in the State.

Costs

Medical errors carry a high financial cost. The IOM report estimates that medical errors cost the Nation approximately $37.6 billion each year; about $17 billion of those costs are associated with preventable errors. About half of the expenditures for preventable medical errors are for direct health care costs.

Not a New Issue

The serious problem of medical errors is not new, but in the past, the problem has not gotten the attention it deserved. A body of research describing the problem of medical errors began to emerge in the early 1990s with landmark research conducted by Lucian Leape, M.D., and David Bates, M.D., and supported by the Agency for Health Care Policy and Research, now the Agency for Healthcare Research and Quality (AHRQ).

The final report of the President's Advisory Commission on Consumer Protection and Quality in the Health Care Industry, released in 1998, identified medical errors as one of the four major challenges facing the Nation in improving health care quality. Based on the recommendations of that report, President Clinton directed the establishment of the Quality Interagency Coordination Task Force (QuIC) to coordinate quality improvement activities in Federal health care programs.

The QuIC includes: the Departments of Health and Human Services, Labor, Veterans Affairs, Commerce, and Defense; the Coast Guard; the Bureau of Prisons; and the Office of Personnel Management. AHRQ Director John M. Eisenberg, M.D., serves as the operating chair of the QuIC.

Public Fears

While there has been no unified effort to address the problem of medical errors and patient safety, awareness of the issue has been growing. Americans have a very real fear of medical errors. According to a national poll conducted by the National Patient Safety Foundation:

• Forty-two percent of respondents had been affected by a medical error, either personally or through a friend or relative.
• Thirty-two percent of the respondents indicated that the error had a permanent negative effect on the patient's health.

Overall, the respondents to this survey thought the health care system was "moderately safe" (rated a 4.9 on a 1 to 7 scale, where 1 is not safe at all and 7 is very safe).

Another survey, conducted by the American Society of Health-System Pharmacists, found that Americans are "very concerned" about:

• Being given the wrong medicine (61 percent).
• Being given two or more medicines that interact in a negative way (58 percent).
• Complications from a medical procedure (56 percent).

Most people believe that medical errors are the result of the failures of individual providers. When asked in a survey about possible solutions to medical errors:

• Seventy-five percent of respondents thought it would be most effective to "keep health professionals with bad track records from providing care."
• Sixty-nine percent thought the problem could be solved through "better training of health professionals."

This fear of medical errors was borne out by the interest and attention that the IOM report generated. According to a survey by the Kaiser Family Foundation, 51 percent of Americans followed closely the release of the IOM report on medical errors.

It's a Systems Problem

The IOM emphasized that most of the medical errors are systems related and not attributable to individual negligence or misconduct. The key to reducing medical errors is to focus on improving the systems of delivering care and not to blame individuals. Health care professionals are simply human and, like everyone else, they make mistakes. But research has shown that system improvements can reduce the error rates and improve the quality of health care:

• A 1999 study indicated that including a pharmacist on medical rounds reduced the errors related to medication ordering by 66 percent, from 10.4 per 1,000 patient days to 3.5 per 1,000 patient days.
• The specialty of anesthesia has reduced its error rate by nearly sevenfold, from 25 to 50 per million to 5.4 per million, by using standardized guidelines and protocols, standardizing equipment, etc.
• One hospital in the Department of Veterans Affairs uses hand-held, wireless computer technology and bar-coding, which has cut overall hospital medication error rates by 70 percent. This system is soon to be implemented in all VA hospitals.

Types of Errors

The IOM defines medical error as "the failure to complete a planned action as intended or the use of a wrong plan to achieve an aim." An adverse event is defined as "an injury caused by medical management rather than by the underlying disease or condition of the patient." Some adverse events are not preventable and they reflect the risk associated with treatment, such as a life-threatening allergic reaction to a drug when the patient had no known allergies to it. However, the patient who receives an antibiotic to which he or she is known to be allergic, goes into anaphylactic shock, and dies, represents a preventable adverse event.

Most people believe that medical errors usually involve drugs, such as a patient getting the wrong prescription or dosage, or mishandled surgeries, such as amputation of the wrong limb. However, there are many other types of medical errors, including:

• Diagnostic error, such as misdiagnosis leading to an incorrect choice of therapy, failure to use an indicated diagnostic test, misinterpretation of test results, and failure to act on abnormal results.
• Equipment failure, such as defibrillators with dead batteries or intravenous pumps whose valves are easily dislodged or bumped, causing increased doses of medication over too short a period.
• Infections, such as nosocomial and post-surgical wound infections.
• Blood transfusion-related injuries, such as giving a patient the blood of the incorrect type.
• Misinterpretation of other medical orders, such as failing to give a patient a salt-free meal, as ordered by a physician.

Preventing Errors

Research clearly shows that the majority of medical errors can be prevented:

• One of the landmark studies on medical errors indicated 70 percent of adverse events found in a review of 1,133 medical records were preventable; 6 percent were potentially preventable; and 24 percent were not preventable.
• A study released last year, based on a chart review of 15,000 medical records in Colorado and Utah, found that 54 percent of surgical errors were preventable.

Other potential system improvements include:

• Use of information technology, such as hand-held bedside computers, to eliminate reliance on handwriting for ordering medications and other treatment needs.
• Avoidance of similar-sounding and look-alike names and packages of medication.
• Standardization of treatment policies and protocols to avoid confusion and reliance on memory, which is known to be fallible and responsible for many errors.

Reducing and Preventing Adverse Drug Events To Decrease Hospital Costs

Adverse drug events (ADEs) result in more than 770,000 injuries and deaths each year and cost up to $5.6 million per hospital, depending on size. Many ADE injuries and resulting hospital costs can be reduced if hospitals make changes to their systems for preventing and detecting ADEs. Some approaches found to be successful are summarized below.

Introduction

Over 770,000 people are injured or die each year in hospitals from adverse drug events (ADEs) , which may cost up to $5.6 million each year per hospital  depending on hospital size. This estimate does not include ADEs causing admissions, malpractice and litigation costs, or the costs of injuries to patients. National hospital expenses to treat patients who suffer ADEs during hospitalization are estimated at between $1.56 and $5.6 billion annually.

Even though research on the cost and causes of ADEs has been reported for years in the medical literature, the problem was brought to the attention of a larger audience in late 1999 by a report by the Institute of Medicine (IOM), To Err Is Human: Building a Safer Health System. This report explores the events surrounding medical errors and the injuries that patients suffer as a result. The IOM concluded that the solution to preventing medical errors is "building a safer health system" that leads health care providers down the appropriate paths of treatment and limits their ability to make mistakes.

Research in this area and the IOM report were based, in part, on studies sponsored by the Agency for Healthcare Research and Quality (AHRQ), the Federal agency charged with sponsoring research to improve the quality, appropriateness, and effectiveness of health care services.

Highlights

• Patients who experienced adverse drug events (ADEs) were hospitalized an average of 8 to 12 days longer than patients who did not suffer ADEs, and their hospitalization cost $16,000 to $24,000 more.
• Anywhere from 28 percent to 95 percent of ADEs can be prevented by reducing medication errors through computerized monitoring systems.
• Computerized medication order entry has the potential to prevent an estimated 84 percent of dose, frequency, and route errors.
• Hospitals can save as much as $500,000 annually in direct costs by using computerized systems.
• During 2001, AHRQ will continue to fund grants designed to reduce medical errors based on the integration of best practices, provider education, and advances in information technology.

Adverse Drug Events Increase Costs

Patient injuries resulting from drug therapy are among the most common types of adverse events that occur in hospitals . Although the incidence of ADEs and their effect on costs have been investigated in only a few hospitals in the United States, the implications are clear from published results that ADEs constitute a widespread problem that causes injuries to patients and disproportionately increases expenses.

Incidence rates of ADEs vary from 2 per 100 admissions to 7 per 100 admissions among the hospitals that have conducted ADE studies. A precise national incidence rate is difficult to calculate because various researchers use different criteria to detect and identify ADEs.

ADEs can result in a number of different physical consequences, ranging from allergic reactions to death. One study estimated that 9.7 percent of ADEs caused permanent disability. Another study estimated that the increased risk of death for a patient who experiences an ADE is nearly twice that of a patient who does not . Figure 1 illustrates the distribution of several types of injuries among patients who suffered ADEs.

Figure 1. Percent patients suffering selected injuries commonly studied among patients who experienced adverse drug events

ADEs and their subsequent injuries lead to increased hospital costs. Depending on facility size, hospital costs annually for all ADEs are estimated to be as much as $5.6 million per hospital. Before the advent of managed care, hospitals would have shifted these costs to the patient or the insurance company. Today, however, hospitals are likely to absorb the extra expense. Patients who experience ADEs have longer, more expensive hospitalizations than patients who do not suffer ADEs. For example, at LDS Hospital in Salt Lake City, researchers found that patients who experienced ADEs were hospitalized an average of 1 to 5 days longer than patients who did not suffer ADEs, with additional costs of up to $9,000.

Researchers conducting an AHRQ-funded study at Brigham and Women's Hospital and Massachusetts General Hospital found that, on average, ADEs increased the length of stay by as much as 4.6 days and increased costs up to $4,685.

These research studies also found that the type of ADE affects length of stay and costs. For example, the costs attributable to ADEs at LDS Hospital ranged from $677 and almost 18 additional hours of hospitalization for itching to $9,022 and 5 1/2 additional days for a drug-induced fever.

A prior study funded by AHRQ at LDS Hospital showed that patients with more severe ADEs (arrhythmia, bone-marrow depression, depression of the central nervous system, seizures, or bleeding) had an average length of stay of 20 days, patients suffering from less severe ADEs (those that required a change in therapy or a longer hospital stay) had an average stay of 13 days, and patients who did not suffer an ADE had an average stay of 5 days. Hospital costs for these patients were $38,007, $22,474, and $6,320, respectively.

Adverse Drug Events Cannot Be Predicted by Patient Characteristics or Drug Type

Anticipating who will suffer an ADE, when, and from what medication is difficult. Research has not yet identified any valid predictors of the event. Patient characteristics currently are not useful predictors of an ADE because patients who have suffered ADEs are not a homogeneous group. Although older age, severity of illness, intensity of treatment, and polypharmacy have been associated with ADEs, no cause and effect relationship is known to exist between patients who suffer ADEs and age, comorbidity, or number of drugs received.  However, ADEs are more likely to result in life-threatening consequences in intensive care unit (ICU) patients than in others.

Medication type is not currently a predictor, either. All drugs have side effects, which are usually discovered during clinical trials required by the Food and Drug Administration (FDA). Because clinical trials are conducted on limited numbers of people under controlled conditions, additional problems are often discovered only after the medication has been prescribed for patients on a routine basis.

For example, in studies funded by AHRQ at LDS Hospital, Brigham and Women's Hospital, and Massachusetts General Hospital, numerous classes of medications were found to be involved in ADEs, including antibiotics (19-30 percent of ADEs), analgesics or pain medications (7-30 percent), electrolyte concentrates (1-10 percent), cardiovascular drugs (8-18 percent), sedatives (4-8 percent), antineoplastic drugs (7-8 percent), and anticoagulants or blood-thinning drugs (1.3-3 percent)  Other classes of medications, such as gastrointestinal drugs, antipsychotics, diabetic medications, antihypertensives, antidepressants, diuretics, hormonal agents, antihistamines, and antiemetics, also account for a small percentage of ADEs.

AHRQ-Funded Research on Medication Errors and Adverse Drug Events

• Clinical Applications of an Expert System, 1989-92, LDS Hospital, Salt Lake City, Utah: Developed a computer system to detect adverse drug events in hospitalized patients.
• Altering Physician Behavior Using Computer Order Entry, 1990-93, Brigham and Women’s Hospital, Boston, Massachusetts: Allowed physicians to order prescription medications using a computer.
• System Changes To Prevent Adverse Drug Events, 1993-94, Brigham and Women’s Hospital and Massachusetts General Hospital, Boston, Massachusetts: Measured the incidence of adverse drug events in hospitalized patients and identified the underlying causes and system failures behind those adverse drug events.
• Assessment of Technology Use Via Computerized Ordering, 1988-93, and Computer Records, Guidelines, Quality, and Efficient Care, 1994-99, Wishard Memorial Hospital, Indianapolis, Indiana: Increased the quality of drug prescribing and helped prevent adverse reactions from pharmaceutical treatments.

Medication Errors Are a Frequent Cause of Adverse Drug Events

Medication errors occur at any point in the medication administration process—during ordering, transcription (the process of manually transferring the physician order onto medication sheets), dispensing, and administering medications . However, as shown in Table 1, the majority of errors occur during the ordering and administration stages.

Table 1. Percent of medication errors occurring within the four stages of the medication process

Physician ordering: 39-49%
Nursing administration: 26-38%
Transcription: 11-12%
Pharmacy dispensing: 11-14%

Sources: (1) Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. JAMA 1995;274(1):29-34. (2) Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995;274(1):35-43.

Figure 2. Commonly studied medication errors as causes of adverse drug events (ADEs): percent of ADEs for each cause

Figure 2  shows the percent for selected types of errors that were commonly associated with ADEs. Other specific errors not shown in Figure 2 that have also been associated with ADEs include:

• Missed dose (7 percent).
• Wrong technique (6 percent).
• Illegible order (6 percent).
• Duplicate therapy (5 percent).
• Drug-drug interaction (3-5 percent).
• Equipment failure (1 percent).
• Inadequate monitoring (1 percent).
• Preparation error (1 percent) .

Adverse Drug Events Can Be Prevented and Detected

Hospital systems can be changed so that ADEs are more readily prevented and detected. Research funded by AHRQ shows that computerized systems can reduce medication errors and prevent ADEs. These studies indicate that anywhere from 28 to 95 percent of ADEs can be prevented.

For example, at least two studies attribute 42-60 percent of ADEs to excessive drug dosage for the patient's age, weight, underlying condition, and renal function. Yet systems are available that prompt doctors to take these factors into consideration when ordering medications.

Even if an ADE is not preventable, computerized systems can detect ADEs early so that health care providers can initiate interventions to mitigate the effects and lessen the severity of the reaction. Hospitals usually rely on hospital staff to complete manual, written incident reports in order to track adverse events, improve quality, and assess risk. However, only a very few (6 percent) of ADEs are reported by this method. Automatic systems can improve detection considerably.

Computer monitoring systems prevent and detect ADEs

Computerized systems currently in use at hospitals perform many different functions. AHRQ has funded research on at least two functions essential to preventing and identifying ADEs—prevention and identification of ADEs and prescription order entry—to determine their effectiveness.

The following discussion of systems at LDS Hospital, Brigham and Women's Hospital, and Wishard Memorial Hospital gives examples of AHRQ-funded research that has focused on how automated systems can improve quality and decrease costs.

LDS Hospital

The HELP (Health Evaluation through Logical Processing) system at LDS Hospital in Salt Lake City identifies patients who may have an ADE through 24-hour monitoring of patient "signals". These signals, routinely recorded in a medical record, are clinical identifiers that indicate an ADE might have occurred:

• Rash.
• Change in respiratory rate, heart rate, hearing, or mental state.
• Seizure.
• Anaphylaxis.
• Diarrhea.
• Fever.

The computer system alerts hospital staff if any of these signals appear or if other signals of possible ADEs occur, such as certain lab test results, high or low blood levels of certain medications, inappropriate medication dosage for the patient's age or weight, and pharmacy orders for medications generally used to treat allergic reactions.

Other findings resulting from an AHRQ-funded grant showed that computer surveillance increased the identification of ADEs. At LDS hospital during 1988-89, ADEs were identified by hospital staff who voluntarily reported ADEs or medication errors. Beginning in 1989, computer monitoring was used to identify ADEs. As a result, significantly more ADEs were identified. LDS was able to isolate the medication errors that were occurring and implement practices that reduced the incidence of ADEs caused by known drug allergies and inappropriate administration. Because new signals were added to the computerized system to recognize ADEs during 1989-90, more ADEs were identified during 1990-91. Composite results from both studies are shown in Table 2.

Table 2. Difference between voluntary and computerized identification of adverse drug events (ADEs) and type of ADE identified at LDS Hospital

Type of ADE: Total ADEs
Voluntary reporting, 1988-89: 9
Computer surveillance, 1989-90: 401
Computer surveillance, 1990-91: 598
Computer surveillance, 1991-92: 529

Type of ADE: Known drug allergies
Voluntary reporting, 1988-89: —
Computer surveillance, 1989-90: 13
Computer surveillance, 1990-91: 0
Computer surveillance, 1991-92: 0

Type of ADE: Inappropriate administration
Voluntary reporting, 1988-89: —
Computer surveillance, 1989-90: 20
Computer surveillance, 1990-91: 1
Computer surveillance, 1991-92: 2

Sources: (1) Evans RS, Pestotnik SL, Classen DC, et al. Prevention of adverse drug events through computerized surveillance. Proc Annu Symp Comput Appl Med Care 1992;437-41. (2) Evans RS, Pestotnik SL, Classen DC, et al. Preventing adverse drug events in hospitalized patients. Ann Pharmacother 1994;28:523-7.

Following AHRQ funding, LDS Hospital also developed a computer-assisted antibiotic-dose monitor that tracks renal function in patients on a daily basis and identifies patients who may be receiving excessive doses of antibiotics . The intervention study conducted with the antibiotic-dose monitor resulted in a reduction of the number, amount, and length of time patients received antibiotics, as well as a reduction in cost, number of days hospitalized, and ADEs, when compared to patients receiving antibiotics before the intervention (Table 3).

Table 3. Comparison of patient antibiotic data before and after implementation of computer-assisted antibiotic-dose monitor at LDS Hospital

Variable: Mean doses
Before intervention: 10.10
After intervention: 8.90

Variable: Mean grams
Before intervention: 9.70
After intervention: 8.70

Variable: Mean cost
Before intervention: $92.96
After intervention: $80.62

Variable: Adverse drug events
Before intervention: 82
After intervention: 14

Variable: Length of stay (days)
Before intervention: 7.0
After intervention: 6.6

Variable: Days of excess dose
Before intervention: 4.7
After intervention: 2.9

Source: Evans RS, Pestotnik SL, Classen DC, et al. Evaluation of a computer-assisted antibiotic dose monitor. Ann Pharmacother 1999;33(10):1026-31.

By integrating the computer system to link pharmacy, lab, and other hospital information about the patient, pharmacists at LDS Hospital were able to identify and notify physicians of drug allergies and drug-drug interactions, as well as drug-food and drug-condition contraindications.

Pharmacists also can monitor the amounts and levels of medication a patient is receiving to prevent ADEs that occur as the result of incorrect or excessive dosages. Notifying physicians immediately of a possible ADE gives them the chance to stop mild drug reactions before they escalate into severe ADEs. At LDS Hospital, when pharmacists notified physicians of an allergic reaction to a drug, 99 percent of the time the physician prescribed a different medication. As a result, during 1990-91, only eight ADEs resulted from allergic reactions.

Brigham and Women's Hospital

The Brigham Integrated Computer System manages all administrative, financial, and clinical information as well as providing clinical-results reporting and computer-based physician order entry (POE). It also incorporates a detection system that identifies different combinations of orders and laboratory results that signal a possible ADE. For example, the system might alert hospital staff to clinical events such as orders for drug antidotes that, along with abnormal laboratory values, indicate a possible ADE. This alert would prompt the staff to further investigate the situation.

Computer monitoring. To test the effectiveness of computer monitoring with regard to identifying ADEs, AHRQ sponsored a study that examined patient records of adult admissions to Brigham and Women's Hospital using three detection methods (computer monitoring, chart review, and voluntary reporting). A total of 617 ADEs were identified, some by more than one method. Although chart review found more ADEs (398) than computer monitoring (275) or voluntary reporting (23), computer monitoring was considered more efficient because it found more ADEs than voluntary reporting and took less time than chart review .

Physician order entry (POE). According to another AHRQ-funded study at Brigham and Women's Hospital, computerized medication order entry has the potential to prevent an estimated 84 percent of dose, frequency, and route errors. Such a system eliminates illegible orders that lead to medication errors. Also, because the system requires the name of the medication, dosage, route, and frequency of administration to be entered, errors that arise from omission of critical information are eliminated. Programmed within the system are algorithms that check dosage frequency, medication interactions, and patient allergies. Once an order is entered, this computerized system also provides physicians with information about the consequences of therapy, benefits, risks, and contraindications.

At Brigham and Women's Hospital, medication error rates and ADE rates were measured before and after implementation of a physician order entry computerized system . The system significantly reduced medication errors and the incidence of ADEs (Table 4). Other studies of the POE system at Brigham and Women's Hospital have shown that the computerized system decreased the incidence of preventable ADEs by at least 17 percent.

Table 4. Number of medication errors and adverse drug event (ADE) rates at Brigham and Women's Hospital before and after introduction of a computer-based physician order entry system*

Medication errors (other than missed dose errors)
Baseline: 242 (142)
Period 1: 134 (51.2)
Period 2: 132 (74.0)
Period 3: 50 (26.6)

Dose errors
Baseline: 81 (47.5)
Period 1: 90(a) (34.3)
Period 2: 114(a) (63.9)
Period 3: 40(a) (21.3)

Frequency errors
Baseline: 43 (25.2)
Period 1: 4 (1.5)
Period 2: 2 (1.1)
Period 3: 4 (2.1)

Route errors
Baseline: 25 (14.7)
Period 1: 5 (1.9)
Period 2: 6 (3.3)
Period 3: 4 (2.1)

Substitution errors
Baseline: 12 (7.0)
Period 1: 3 (1.1)
Period 2: 3 (1.7)
Period 3: 0 (0)

Documented allergy
Baseline: 10 (5.9)
Period 1: 1 (0.4)
Period 2: 1 (0.6)
Period 3: 0 (0)

Inappropriate drug
Baseline: 7 (4.1)
Period 1: 3 (1.1)
Period 2: 1 (0.6)
Period 3: 0 (0)

Avoidable delay
Baseline: 7 (4.1)
Period 1: 0 (0)
Period 2: 0 (0)
Period 3: 0 (0)

Drug-drug interaction
Baseline: 2 (1.2)
Period 1: 0 (0)
Period 2: 1 (0.6)
Period 3: 0 (0)

Inadequate followup
Baseline: 1 (0.6)
Period 1: 0 (0)
Period 2: 0 (0)
Period 3: 0 (0)

Other
Baseline: 54 (31.7)
Period 1: 28 (10.7)
Period 2: 4 (2.2)
Period 3: 2 (1.1)

Total ADEs
Baseline: 25 (14.7)
Period 1: 39 (14.9)
Period 2: 19 (10.7)
Period 3: 18 (9.6)

Intercepted potential ADEs
Baseline: 27 (15.8)
Period 1: 82(a) (31.3)
Period 2: 106(a) (59.4)
Period 3: 1 (0.5)

Non-intercepted potential ADEs
Baseline: 8 (4.7)
Period 1: 4 (1.5)
Period 2: 1 (0.6)
Period 3: 0 (0)

Note: The figure in parentheses indicates the rate per 1,000 patient days.

(a) Seventy-seven dose errors in period 1 and 101 in period 2 were due to potassium chloride errors; none of 40 dose errors in period 3 were due to potassium chloride error. After revising potassium chloride ordering screens, these errors were eliminated.

Source: Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Inform Assoc 1999;6(4):313-21.

Wishard Memorial Hospital

AHRQ has funded a number of studies at Wishard Memorial Hospital to support development of the Regenstrief Medical Record System (RMRS). The RMRS computer database contains a patient's entire medical record, including inpatient and outpatient data for Wishard's patients.

One AHRQ-sponsored study examined whether or not automatically providing suggested "corollary" orders when physicians prescribe certain drugs electronically would reduce certain errors. For example, if a patient is receiving heparin (a blood thinner), a corollary order would be to order an initial platelet count and then another after 24 hours to determine if the patient is receiving too much or not enough medication. Another example of a corollary order is for tests that monitor electrolytes in a patient receiving a potassium supplement. The investigators predicted the incidence of ADEs would be reduced if these corollary orders were prompted.

Results from this study showed that physicians who were offered corollary orders had the suggested tests done at twice the rate of physicians who were not offered the corollary orders (46.3 percent compared to 21.9 percent). As predicted, the incidence of ADEs was also reduced. The study also found that drug-related hospital incident reports were one-third lower for patients whose physicians wrote orders using computerized order-writing workstations than for patients whose doctors used paper forms.

Good Samaritan Regional Medical Center

Other research substantiates that computer systems can help reduce and prevent ADEs. A computer alert system at Good Samaritan Regional Medical Center was designed to detect and alert health care providers to prescription errors and ADEs. The computer database contained patient information such as demographics, pharmacy orders, drug allergies, radiology orders, and lab results .

A series of primary prevention alerts identified possible prescription errors and recommendations for correction. For example, if lab tests revealed a low potassium or magnesium level or a high digoxin level for a patient prescribed digoxin, the computer recommended electrolyte replacement or reduction of digoxin dose. Secondary prevention alerts were programmed to indicate a potential ADE. For example, if a patient were suffering from delirium, the computer would prompt the pharmacist to review all medications to see if the delirium was drug induced .

The results of this study (Table 5) indicate that the computer alert system prevented and detected ADEs that otherwise would not have been recognized.

Table 5. Results of computer alert system for detection of medication errors and adverse drug events (ADEs) at Good Samaritan Regional Medical Center

Alert type: Detection of prescription error
Total alerts: 803
True-positive alerts: 490
Unrecognized by physician prior to alert: 238 (49%)

Alert type: Early detection of ADE
Total alerts: 313
True-positive alerts: 106
Unrecognized by physician prior to alert: 27 (25%)

Note: A true-positive alert was defined as one in which the physician wrote orders consistent with the alert recommendation after alert notification.

Source: Raschke RA, Collihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events. JAMA 1998;280(15):1317-20.

Improved systems can save direct costs

In 1992, 567 ADEs cost LDS Hospital $1.1 million in direct expenses (not including liability costs or the cost of injuries to patients). If 50 percent of these ADEs had been prevented, LDS Hospital would have saved $500,000 a year . Brigham and Women's Hospital would have saved $480,000 annually if the 17 percent decrease in ADEs had been applied hospital-wide; this figure does not include the costs to patients of injuries, malpractice costs, or the expense of additional work required to correct medication errors and treat patients who suffer from ADEs . Doctors at Wishard Memorial Hospital who used computer order-writing workstations had 13 percent lower inpatient charges (about $900 per hospitalization) than those who used paper forms.

Tracking ADE occurrences to discover trends can save money also. At LDS Hospital, ADE trending identified 25 ADEs related to a new brand of vancomycin. This brand was being used because it cost $5,000 a year less than a brand they had previously used. However, treating the patients who suffered these ADEs cost $50,000 in extra care expenses. Thus, without tracking, the hospital would have assumed it was saving $5,000 a year, whereas switching brands actually cost the hospital $45,000.

Other Systems Can Prevent and Reduce Adverse Drug Events

Computer systems are only part of the solution in preventing and reducing ADEs. Research studies (some funded in part by AHRQ) on medication errors support other methods that improve the medication delivery system. These include:

• Using the FDA's MedWatch program to report serious adverse drug reactions. Reporting would allow the FDA to pass safety information on to other providers, require labeling changes, or even withdraw a drug from the market. MedWatch reports can be submitted through the FDA's Web site at www.fda.gov/medwatch/ or by calling MedWatch at 1-800-332-1088 to obtain the necessary forms .
• Improving incident reporting systems. The process of incident reporting can be streamlined to accommodate the health care provider's busy schedule and can offer feedback indicating that reported information is being used. Health care workers sometimes do not recognize that a change in a patient's condition is due to pharmaceutical treatment. Therefore, workers should be educated to identify signs and symptoms that might indicate an ADE and thus increase reporting of ADEs .
• Creating a better atmosphere for health care providers to report ADEs where the person reporting the error does not fear repercussions or punishment . As the aviation industry has discovered, punishment is a deterrent to reporting an error; if an error is not reported, nothing can be done to correct the situation that created the potential for error. Health care personnel can find it difficult to acknowledge that they make mistakes.
• Relying more on pharmacists to advise physicians in prescribing medications, and promoting health care provider education on medications. Brigham and Women's Hospital reduced the ADE rate in its Intensive Care Unit (ICU) from 33.0 per 1,000 patient days to 11.6 per 1,000 patient days by having a pharmacist participate in patient rounds with the ICU team. As a result, the hospital estimated it could reduce its costs by $270,000 per year simply by using the pharmacist's time in a different manner.
• Improving the nursing medication administration and monitoring systems . These changes might include bar coding medications, along with additional warnings on medications with higher potential for harm, such as insulin, opiates, narcotics, potassium chloride, and anticoagulants.

Translating Research Into Practice

Reducing Errors in Health Care

Medical errors are responsible for injury in as many as 1 out of every 25 hospital patients; an estimated 48,000-98,000 patients die from medical errors each year. Errors in health care have been estimated to cost more than $5 million per year in a large teaching hospital, and preventable health care-related cost the economy from $17 to $29 billion each year.

AHRQ research has shown that medical errors may result most frequently from systems errors— organization of health care delivery and how resources are provided in the delivery system.

Patients at Risk

Medical errors may result in:

• A patient inadvertently given the wrong medicine.
• A clinician misreading the results of a test.
• An elderly woman with ambiguous symptoms (shortness of breath, abdominal pain, and dizziness) whose heart attack is not diagnosed by emergency room staff.

Errors like these are responsible for preventable injury in as many as 1 out of every 25 hospital patients.

Errors in health care have been estimated to cost more than $5 million per year in a large teaching hospital.  According to a recent report by the Institute of Medicine (IOM) , preventable health care-related injuries cost the economy from $17 to $29 billion annually, of which half are health care costs.

The IOM report  estimates that 44,000 to 98,000 people each year die from medical errors. Even the lower estimate is higher than the annual mortality from motor vehicle accidents (43,458), breast cancer (42,297), or AIDS (16,516), thus making medical errors the eighth leading cause of death in the United States.

These and other findings of the IOM report are based on research sponsored by a variety of organizations, including the Agency for Healthcare Research and Quality (AHRQ).

For example, a study by AHRQ  found that just one type of error—preventable adverse drug events—caused one out of five injuries or deaths per year to patients in the hospitals that were studied.

How Errors Occur

Errors can occur at any point in the health care delivery system, AHRQ-supported research has revealed.

Medication Errors

These are preventable mistakes in prescribing and delivering medication to patients, such as prescribing two or more drugs whose interaction is known to produce side effects or prescribing a drug to which the patient is known to be allergic.

Research by AHRQ-supported investigators is helping to characterize these errors (called preventable adverse drug events, or ADEs) and suggest how to prevent them.

• In a study of inpatient care in two tertiary care hospitals, errors in ordering and administering medicines accounted for 56 and 34 percent, respectively, of preventable adverse drug events.
• Findings from a second study  showed that dosage errors, in particular, were primarily due to the physician's lack of knowledge about the drug or about the patient for whom it was prescribed.
• An attempt to identify risk factors for preventable adverse drug reactions among patients admitted to medical and surgical units at two large hospitals  found few such factors, which suggested to the researchers that a focus on improving medication systems would prove more effective.

Surgical Errors

In contrast to ADEs, surgical adverse events (1 in 50 admissions in Colorado and Utah hospitals during 1992) , accounted for two-thirds of all adverse events and 1 of 8 hospital deaths in a recent retrospective study of these institutions by an AHRQ fellow.

Diagnostic Inaccuracies

Incorrect diagnoses may lead to incorrect and ineffective treatment or unnecessary testing, which is costly and sometimes invasive. Also, inexperience with a technically difficult diagnostic procedure can affect the accuracy of the results. Here, too, AHRQ-funded researchers have made major contributions.

• One study  showed that physicians who performed 100 or more colposcopies (a test used to follow up abnormal Pap smears) a year had more accurate findings than physicians who performed the procedure less often.
• Another study  demonstrated that measuring blood pressure with the most commonly used type of equipment often gives incorrect readings that may lead to mismanagement of hypertension.

System Failures

Although errors in medication, surgery, and diagnosis are the easiest to detect, medical errors may result more frequently from the organization of health care delivery and the way that resources are provided to the delivery system. Research by AHRQ-supported scientists is helping to identify the systemic factors contributing to preventable adverse events.

• Investigators in a major study  discovered that failures at the system level were the real culprits in over three-fourths of adverse drug events.
• Failures in disseminating pharmaceutical information, in checking drug doses and patient identities, and in making patient information available are system errors that accounted for adverse drug events in over half of the hospitals studied.
• One system-level factor, staffing levels of nurses (adjusted for hospital characteristics), was found in a study  to influence the incidence of adverse events following major surgery, such as urinary tract infections, pneumonia, thrombosis, and pulmonary compromise.

This research on systemic problems leads investigators to conclude that any effort to reduce medical errors in an organization requires changes to the system design, including possible reorganization of resources by top-level management.

Improving Patient Safety

Research funded by AHRQ and others has been important in identifying the extent and causes of errors. Now, additional research is needed to develop and test better ways to prevent errors, often by reducing the reliance on human memory. Some areas of past research that have shown promise in helping to reduce errors include computerized ADE monitoring, computer-generated reminders for follow up testing, and standardized protocols.

Computerized ADE Monitoring

Although chart review was found in an AHRQ-funded study  to be more accurate than computer tracking and voluntary reporting in identifying adverse drug events, it required five times more personnel time. Researchers concluded that the computerized method was the most efficient means of tracking drug errors.

Computer-Generated Reminders for Followup Testing

Some diagnostic tests must be repeated to follow up certain conditions, but a small number of such repeat tests are done too early to yield useful results. In contrast, laboratory results showing that a patient needs critical care may not be communicated in a timely manner.

• One study funded by AHRQ  found that a computerized reminder system to alert physicians to the proper timing of repeat tests reduced the number of patients who were subjected to unnecessary repeat testing.
• The same research group subsequently reported  that an automatic alerting system for communicating critical laboratory results reduced the time until appropriate treatment when compared with the existing hospital paging system.

Standardized Protocols

An AHRQ-sponsored study of patients in intensive care units who had severe respiratory disease found a four-fold increase in survival rate with the use of computerized treatment protocols.

Still other investigators are testing computerized decision support systems in various patient populations. All of these research efforts reflect AHRQ's commitment to improving patient safety by providing new tools to augment provider judgment.

AHRQ-funded research continues to create and test methods to help clinicians avoid errors in health care delivery. An investigation funded by AHRQ and the National Institute on Aging will address the incidence and preventability of adverse drug events in elderly patients receiving ambulatory care.

The Agency has recently funded four Centers for Education and Research in Therapeutics (CERTs)  as part of a 3-year demonstration program. The CERTs will conduct research to increase understanding of ways to improve the appropriate and effective use of drugs, biologicals, and devices in treatments and to avoid adverse events. These centers will also add to our knowledge of the possible risks of new uses of drugs, and combinations of drugs, as they are prescribed in everyday practice.

In addition, the Agency has recently announced  that it will enter into cooperative agreements with nonprofit and for-profit health care organizations to test the effectiveness of the transfer and application of systems-based best practices to reduce medical errors and improve patient safety. This research will help identify high-risk patients or patient groups, providers, health care processes and settings, as well as developing generalizable methods for error reduction.

Promoting Safety

AHRQ (then known as AHCPR, the Agency for Health Care Policy and Research) supported the conference "Enhancing Patient Safety and Reducing Errors in Health Care," which launched the National Patient Safety Foundation.

AHRQ also works with partners, such as the National Committee on Patient Information and Education (NCPIE), to promote patient awareness of medication safety. In 1997, AHCPR and NCPIE co-sponsored the publication of a consumer guide, Prescription Medicines and You, to help consumers understand how to avoid errors in taking medicines.

In sum, AHRQ's contributions have resulted in a broader understanding of the nature of patient safety problems and where they occur in the delivery of health care. AHRQ-supported research is in the forefront of a rethinking of health care systems to reduce medical errors.

Fact Sheet

Patient Safety Reporting Systems and Research in HHS

Reporting is an important component of systems to improve patient safety. Many States have mandatory reporting systems for hospitals, but it is becoming clear that some means of integrating these and other patient safety data is needed to allow researchers to identify sources of problems and their solutions.

The Department of Health and Human Services (HHS) has a number of existing or developing reporting systems and databases that are candidates for integration. These systems are described in this fact sheet, as are other HHS efforts to understand patient safety data needs.

Agency for Healthcare Research and Quality (AHRQ)

AHRQ's Fiscal Year 2001 patient safety and medical error reduction agenda is being accomplished through a series of grants and contracts to stimulate research and demonstration projects. AHRQ will support a portfolio of projects to test the effectiveness, costs, and cost-effectiveness of diverse reporting strategies and information technology innovations on the identification, management, and reduction of medical errors.

Specifically, an AHRQ program entitled "Improving Patient Safety: Health System Reporting, Analysis, and Safety Improvement Research Demonstrations" will support large demonstration projects in States, health care systems, and networks of providers (both integrated delivery systems and primary care networks) to test reporting strategies and to develop patient safety interventions.

Centers for Disease Control and Prevention (CDC)

National Nosocomial Infections Surveillance (NNIS) System

The NNIS System is a voluntary reporting system, started cooperatively by the National Center for Infectious Diseases, CDC, and participating acute-care hospitals to create a national database of hospital-associated infections. The database is used to describe the epidemiology of these infections, describe antimicrobial resistance trends in hospitals, and produce hospital-associated infection rates to use for comparison purposes. The data are used to track progress nationally in reducing infections in hospitalized, high-risk patients and are used by participating and nonparticipating hospitals to emulate the methods to detect problems and monitor prevention and control efforts.

By law, CDC assures participating hospitals that any information that would permit identification of any individual or institution will be held in strict confidence. Trained infection control personnel use standardized definitions and surveillance protocols to ensure that the data are uniformly collected.

National Center for Health Statistics (NCHS) data systems to monitor patient safety

NCHS data can be used to examine questions of patient safety by profiling issues such as:

• Complications and adverse events in hospital settings.
• Avoidable hospitalizations, conditions, or procedures.
• Visits to emergency departments or outpatient settings for adverse effects of medication or medical misadventure.
• Underuse of appropriate medications and treatments.
• Adverse events, such as falls, fractures, or pressure ulcers in nursing-home or home and hospice care settings.
• Deaths due to medical misadventure.
• Characteristics of the health care system that may be related to safety.
• Characteristics of patients experiencing errors.

National Immunization Program—Vaccine Adverse Events Reports System (VAERS)

VAERS is a passive, postmarketing surveillance system to monitor rare adverse events associated with vaccination. Vaccine adverse events are received through VAERS, which is primarily managed by the CDC. Reports are forwarded to Food and Drug Administration (FDA).

FDA

The FDA currently operates reporting systems and databases within three organizational components:

• The Center for Drug Evaluation and Research (CDER).
• The Center for Devices and Radiological Health (CDRH).
• The Center for Biologics Evaluation and Research (CBER).

CDER

MedWatch—FDA Medical Products Reporting Program
MedWatch is a voluntary medical products reporting program. Health care professionals and consumers directly report adverse events involving medical products and product problems, including medical product errors, to this confidential and protected passive surveillance system. All reports undergo triage and are transferred to the appropriate Center for evaluation and entry into the appropriate FDA database.

Adverse Events Reporting System (AERS)
Mandatory reports of adverse events involving drugs and therapeutic biologic products, including events attributable to a medication error, are received from manufacturers and distributors. They must submit expedited reports, for serious and unexpected adverse events; quarterly or annual periodic reports must be submitted for all other serious and nonserious events.

Voluntary and mandatory reports are entered into the AERS, which assigns an individual safety-report identification number for each report. All data undergo data-entry quality control to ensure completeness and accuracy. The events are coded by use of a standardized international terminology (the Medical Dictionary for Regulatory Activities), and the reports are routed directly to assigned clinical reviewers.

CDRH

Manufacturer and User Device Experience (MAUDE) Database—Medical Devices
Voluntary reports are received via MEDWATCH.  Mandatory reports for device-related adverse events are received from medical device manufacturers. In addition, user facilities are required to report device-related serious injuries to the manufacturer and device-related deaths to the manufacturer and directly to FDA. Voluntary and mandatory reports are entered into the MAUDE database.

Medical Product Surveillance Network (MedSuN) Pilot Program—Medical Devices
FDA is required to explore options for designing a national surveillance system based on a representative sample of medical device user facilities. This new reporting system, called MedSuN, will eventually replace the mandatory reporting by all user facilities of medical device-related deaths and serious injuries. Only those facilities selected to be in the program will be under obligation to report to CDRH.

CBER

Biological Product Deviation (BPD) Reporting System
Licensed manufacturers of all biological products, unlicensed registered blood establishments, and transfusion services are required to report any event associated with biologics—including blood, blood components, and source plasma—that represents a deviation in manufacturing. These events include deviations from current good manufacturing practices, applicable regulations, applicable standards, or established specifications that may affect the safety, purity, or potency of a distributed product.

The reports must be submitted as soon as possible, but not more than 45 calendar days from the date the manufacturer acquires information on a reportable event. Quarterly and annual summary reports are prepared from these data, including the total number of reports submitted to FDA over a period of time, categorized by the types of establishments reporting, the types of events reported, and how quickly the reports are submitted.

FDA Collaborative Reporting Systems/Databases

Medication Errors Reporting System (MERS)
FDA receives reports of medication errors directly from the US Pharmacopoeia (USP). The MERS program is operated by the USP in cooperation with the Institute for Safe Medication Practices and is a voluntary and confidential reporting program for health care practitioners to report medication errors. This program can be accessed by mail, telephone, or Internet.

Medical Event Reporting System for Transfusion Medicine (MERS-TM)
The National Heart, Lung, and Blood Institute, in cooperation with FDA, is developing and pilot testing MERS-TM. The system includes software to assist in the detection, selection, description, classification, and analysis of transfusion events, including both errors and near-misses. MERS-TM uses a well-defined set of codes for event reporting, including event description and root-cause analysis.

References

1. Brennan TA, Leape LL, Laird NM, et al. Incidence of adverse events and negligence in hospitalized patients: Results of the Harvard Medical Practice Study I. N Engl J Med 1991;324:370-6.

2. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug events in hospitalized patients. JAMA 1997;277(4):307-11.

3. Institute of Medicine. To Err is Human: Building a Safer Health System. Washington, D.C.: National Academy Press; 1999.

4. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377-84.

5. Bates D, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. JAMA 1995;274(1):29-34.

6. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995; 274(1):35-43.

7. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. Arch Intern Med 1999;159:2553-60.

8. Gawande AA, Thomas EJ, Zinner MJ, et al. The incidence and nature of surgical adverse events in Colorado and Utah in 1992. Surgery 1999;126(1):66-75.

9. Gordon P. Diagnostic accuracy of community physicians performing colposcopy. AHCPR Grant HS07162 Final Report; 1996.

10. Hla KM. Impact of errors in blood pressure measurement. AHCPR Grant HS07301 Final Report; 1994.

11. Kovner C, Gergen PJ. Nurse staffing levels and adverse events following surgery. Image J Nurs Sch 1998;30(4):315-21.

12. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: Development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5(3):305-14.

13. Bates DW, Kuperman GJ, Rittenberg E, et al. A randomized trial of a computer-based intervention to reduce utilization of redundant laboratory tests. Am J Med 1999;106(2):144-50.

14. Kuperman GJ, Teich JM, Tana sijevic MJ, et al. Improving response to critial laboratory results with automation: Results of a randomized controlled trial. J Am Med Inform Assoc 1999;6(6):512-22. Abstract.

15. Morris AH. Protocol management of adult respiratory distress. New Horizons 1993;1(4):593-602.

16. AHCPR launches research program to improve the safe and effective use of medical products. [News story] Research Activities Oct 1999;230:15.

References

1. Classen DC, Pestotnik SL, Evans RS, et al. Adverse drug events in hospitalized patients. JAMA 1997;277(4):301-6.

*2. Cullen DJ, Sweitzer BJ, Bates DW, et al. Preventable adverse drug events in hospitalized patients: A comparative study of intensive care and general care units. Crit Care Med 1997;25(8):1289-97.

*3. Cullen DJ, Bates DW, Small SD, et al. The incident reporting system does not detect adverse drug events: A problem for quality improvement. Journal on Quality Improvement 1995;21(10): 541-8.

*4. Bates DW, Spell N, Cullen DJ, et al. The costs of adverse drug events in hospitalized patients. JAMA 1997;277(4):307-11.

*5. Bates DW, Cullen DJ, Laird N, et al. Incidence of adverse drug events and potential adverse drug events. JAMA 1995;274(1):29-34.

6. Raschke RA, Collihare B, Wunderlich TA, et al. A computer alert system to prevent injury from adverse drug events. JAMA 1998;280(15):1317-20.

7. Thomas EJ, Studdert DM, Newhouse JP, et al. Costs of medical injuries in Utah and Colorado. Inquiry 1999;36(3):255-64.

8. Institute of Medicine. To Err is Human: Building a Safer Health System. Washington: National Academy Press; 1999.

9. Leape LL, Brennan TA, Laird N, et al. The nature of adverse events in hospitalized patients. Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324(6):377-84.

*10. Jha AK, Kuperman GJ, Teich JM, et al. Identifying adverse drug events: Development of a computer-based monitor and comparison with chart review and stimulated voluntary report. J Am Med Inform Assoc 1998;5(3):305-14.

*11. Bates DW, Boyle DL, Vander Vliet MB, et al. Relationship between medication errors and adverse drug events. J Gen Intern Med 1995;10(4):199-205.

12. General Accounting Office (U.S.). Adverse Drug Events. GAO/HEHS-00-21; Jan 2000.

*13. Evans RS, Pestotnik SL, Classen DC, et al. Prevention of adverse drug events through computerized surveillance. Proc Annu Symp Comput Appl Med Care 1992:437-41.

14. Thomas EJ, Studdert DM, Burstin HR, et al. Incidence and types of adverse events and negligent care in Utah and Colorado. Med Care 2000;38(3):261-71.

15. Evans RS, Classen DC, Stevens LE, et al. Using a hospital information system to assess the effects of adverse drug events. Proc Annu Symp Comput Appl Med Care 1993:161-5.

*16. Bates DW, Miller EB, Cullen DJ, et al. Patient risk factors for adverse drug events in hospitalized patients. Arch Intern Med 1999;159(21):2553-60.

17. Food and Drug Administration (U.S). From test tube to patient: Improving health through human drugs. DHHS Publication No. (FDA) 99-3168. Rockville (MD): Food and Drug Administration;1999.

18. Lesar TS, Briceland L, Stein DS. Factors related to errors in medication prescribing. JAMA 1997;277(4):312-7.

*19. Leape LL, Bates DW, Cullen DJ, et al. Systems analysis of adverse drug events. JAMA 1995;274(1):35-43.

*20. Evans RS, Pestotnik SL, Classen DC, et al. Preventing adverse drug events in hospitalized patients. Ann Pharmacother 1994;28(4):523-7.

21. Evans RS, Pestotnik SL, Classen DC, et al. Evaluation of a computer-assisted antibiotic dose monitor. Ann Pharmacother 1999;33(10):1026-31.

22. Bates DW, Teich JM, Lee J, et al. The impact of computerized physician order entry on medication error prevention. J Am Med Inform Assoc 1999;6(4):313-21.

23. Bates DW, Leape LL, Cullen DJ, et al. Effect of computerized physician order entry and a team intervention on prevention of serious medication errors. JAMA 1998;280(15):1311-16.

*24. Overhage JM, Tierney WM, Zhou X, et al. A randomized trial of "corollary orders" to prevent errors of omission. J Am Med Inform Assoc 1997;4(5):364-75.

*25. Tierney WM. Adverse drug events synthesis review—AHRQ. Personal correspondence with Christine Williams; 2001.

26. Sexton JB, Thomas EJ, Helmreich RL. Error, stress, and teamwork in medicine and aviation: Cross-sectional surveys. BMJ 2000;320(7237):745-9.

27. Leape LL, Cullen DJ, Clapp MD, et al. Pharmacist participation on physician rounds and adverse drug events in the intensive care unit. JAMA 1999;282(3):267-70.

28. Schaubhut RM, Jones C. A systems approach to medication error reduction. J Nurs Care Qual 2000;14(3):13-27.

29. Eisenberg JM. The best offense is a good defense against medical errors: Putting the full-court press on medical errors. Speech given by Director, Agency for Healthcare Research and Quality, at the Duke University Clinical Research Institute; 2000 Jan 2.

30. Eisenberg JM. Statement on medical errors given by Director, Agency for Healthcare Research and Quality, before the Senate Appropriations Subcommittee on Labor, Health and Human Services, and Education; 1999 Dec 13.

31. Agency for Heath Care Policy and Research launches research program to improve the safe and effective use of medical products. Press release; 1999 Sep 29.

32. Centers for Education and Research on Therapeutics: Overview. October 2000. Agency for Healthcare Research and Quality, Rockville, MD.