All authors have completed the Unified Competing Interest form at
Conceived and designed the experiments: GBR CDM EO MS SD. Performed the experiments: GBR AG SD. Analyzed the data: GRB IHZ LDW SD. Wrote the paper: GBR CDM IHZ EO AG MS SD. Obtaining funding: MS SD Administrative, technical, or material support: GBR CDM AG LDW MS SD Study supervision: GBR AG SD.
Over 50% of antibiotics prescriptions are for outpatients with acute respiratory infections (ARI). Many of them are not needed and thus contribute both avoidable adverse events and pressures toward the development of bacterial resistance. Could a clinical decision support system (CDSS), interposed at the time of electronic prescription, adjust antibiotics utilization toward consensus treatment guidelines for ARI?
This is a retrospective comparison of pre- (2002) and post-intervention (2003–2006) periods at two comprehensive health care systems (intervention and control). The intervention was a CDSS that targeted fluoroquinolone and azithromycin; other antibiotics remained unrestricted. 7000 outpatients visits flagged by an ARI case-finding algorithm were reviewed for congruence with the guidelines (antibiotic prescribed-when-warranted or not-prescribed-when-unwarranted).
3831 patients satisfied the case definitions for one or more ARI: pneumonia (537), bronchitis (2931), sinusitis (717) and non-specific ARI (145). All patients with pneumonia received antibiotics. The relative risk (RR) of congruent prescribing was 2.57 (95% CI = (1.865 to 3.540) in favor of the intervention site for the antibiotics targeted by the CDSS; congruence did not change for other antibiotics (adjusted RR = 1.18 (95% CI = (0.691 to 2.011)). The proportion of unwarranted prescriptions of the targeted antibiotics decreased from 22% to 3%, pre vs. post-intervention (p<0.0001).
A CDSS interposed at the time of e-prescription nearly extinguished unwarranted use targeted antibiotics for ARI for four years. This intervention highlights a path toward sustainable antibiotics stewardship for outpatients with ARI.
Microorganisms resistant to antibiotics increase the mortality, morbidity and costs of infections. Without a drug development infrastructure that can keep pace with the rapidly evolving resistance mechanisms, these organisms are expected to threaten public health for years to come.
Because exposure to antibiotics is a key promoter of bacterial resistance
In this work, we re-engineered pharmacy processes to interpose a clinical decision support systems (CDSS) at the time of order entry for selected antibiotics
This is a retrospective, observational study designed to assess the effect of a CDSS on congruence of antibiotics prescribing with widely endorsed ARI treatment guidelines
The Institutional Review Boards of the participating VA health systems, the University of Maryland and the University of Utah, approved the study. The study was granted a waiver of consent as risks were limited to information confidentiality and it involved a large number of participants.
The CDSS intervention was part of a larger quality improvement initiative that targeted 26 medications and was used by at least 1379 unique providers during the study period. The CDSS: 1) deployed drug-specific guideline recommendations as clickable choices during order entry; 2) mined the electronic medical record (EMR) for patient/context specific information; and 3) based on what the provider had clicked on, issued a note documenting the rationale for drug use. The providers could then accept or modify this note before committing it to the EMR. The presence of this note was verified by pharmacy, but its content was not routinely audited.
The azithromycin and gatifloxacin CDSS included treatment paths for the following diseases: community-acquired pneumonia, acute bronchitis, acute sinusitis, non-specific upper respiratory infection (URI) and exacerbations of chronic obstructive pulmonary disease (COPD). An “Other” path provided access to either drug for provider-supplied indications.
For the community-acquired pneumonia path, providers clicked on the diagnostic elements that raised their index of suspicion for pneumonia and were then led to a prescription. For the acute bronchitis
A case-detection algorithm previously found to identify 76% of patients with an influenza-like illness
With the exception of notes generated by the CDSS itself, all free-text EMR entries on the day of flagged visits were manually abstracted for data elements needed to assign ARI diagnoses and treatment. Reviewers cross-validated 10% of each other’s work; conflicts were resolved through arbitration with a pulmonary medicine specialist. Inter-rater reliability was determined using 15% of the total sample for the “cough”, “sputum production”, “cough duration” and “sputum production duration” symptoms (kappa statistics = 0.80, 0.87, 0.87 and 0.87 respectively at the intervention site and 0.86, 0.79, 0.91, and 0.92 respectively at the control site). Structured EMR data elements, such as prescriptions and vital signs, were extracted from the EMR
We reviewed all of the records flagged by the case-detection algorithm at the intervention site. Based on preliminary results and published effect sizes
Of the 7000 visits manually reviewed, 3169 were excluded according to pre-defined criteria: 1) not an outpatient (n = 141); 2) not an ARI (n = 855); 3) not an in-person, initial visit for a given ARI episode (no prior ARI visit within 3 weeks) (n = 1093); 4) prior ARI episode(s) during the study period i.e. patients were used only once (n = 140); 5) stated diagnosis of COPD, whether or not the visit was related to an exacerbation of this disease (n = 501); 6) acute pharyngitis as the only ARI diagnosis (n = 431).
A visit was labeled pneumonia if a provider note listed this diagnosis as likely. Other ARI case definitions and conditions justifying antibiotics matched that of the guidelines (
ARI Condition | Diagnostic Criteria | Antibiotic Treatment Criteria |
Pneumonia | Clinician’s documented diagnostic impression | Antibiotics always warranted |
Acute Bronchitis | 1) Acute cough (productive or not) 2) Cough duration <21 days | Antibiotics not warranted |
Pharyngitis | 1) Sore throat 2) Erythematous throat | At least three of the following four symptoms/signs: 1) History of fever or temperature >100.4°F (38°C); 2) Tonsillar exudate; 3) Tender anterior cervical lymphadenopathy; 4) Absence of cough |
Sinusitis | 1) Purulent nasal discharge2) Facial or sinus pain 3) Sinus tenderness4) Productive cough | Severe symptoms or symptom duration ≥7 days including purulent nasal discharge/drainage AND (Maxillary facial or tooth (sinus) pain OR tenderness) |
Non-Specific Acute Upper Respiratory Infection | 1) Absence of a predominant sinus, pharyngeal or lower airway symptom 2) Nasal discharge 3) Sputum production from the throat | Antibiotics not warranted |
Descriptive statistics and results from bivariate tests for association were generated for the study sample and informed the multivariable analyses. Multivariable logistic regression and difference-in-difference regression analyses using a Poisson distribution with a log link and modified Poisson approach to obtain robust standard error estimates
The study included 3831 unique patients with an initial visit for ARI. Patients were mostly older males (
Characteristics | Intervention Site N (%) | Control SiteN (%) |
Sample Size | 2669 | 1162 |
Sex | ||
Male | 2439 (91.4) | 1096 (94.32) |
Female | 230 (8.6) | 66 (5.68) |
Self-Reported Race | ||
African American | 1775 (66.5) | 18 (1.55) |
White | 601 (22.5) | 697 (59.98) |
Latino | 17 (0.6) | 41 (3.53) |
Other | 10 (0.4) | 14 (1.20) |
Missing | 266 (10.0) | 392 (33.73) |
Marital Status | ||
Married | 792 (29.7) | 620 (53.36) |
Unmarried | 1877 (70.3) | 542 (46.64) |
Age at Encounter Date, years | ||
Mean | 55.6 (13.9) | 59.1 (15.55) |
Median | 53 | 58 |
Range | 16–97 | 19–91 |
ARI Visits, by year | ||
|
||
2002 | 373 (14.0) | 344 (29.60) |
|
||
2003 | 673 (25.2) | 253 (21.77) |
2004 | 481 (18.0) | 185 (15.92) |
2005 | 770 (28.9) | 225 (19.36) |
2006 | 372 (13.9) | 155 (13.34) |
ARI Visits, by Condition(s) | ||
Pneumonia | 337 (12.6) | 200 (17.2) |
Bronchitis Only | 941 (35.3) | 292 (25.1) |
Sinusitis Only | 57 (2.1) | 66 (5.7) |
Bronchitis plus pharyngitis | 918 (34.4) | 281 (24.2) |
Bronchitis plus sinusitis | 124 (4.6) | 109 (9.4) |
Bronchitis plus pharyngitis plus sinusitis | 139 (5.2) | 127 (10.9) |
inusitis plus pharyngitis | 51 (1.9) | 44 (3.8) |
on-specific URI | 102 (3.8) | 43 (3.7) |
Counts of ARI Patients and Visits by Characteristics and Sites over the whole study period (2002–2006, n = 3831).
Of the 624 visits where antibiotics were warranted (
Intervention Site | Control Site | |||||||
Antibiotics | Warranted | Unwarranted | Warranted | Unwarranted | ||||
2002 | 2003–6 | 2002 | 2003–6 | 2002 | 2003–6 | 2002 | 2003–6 | |
Prescribed (Targeted) | 26 | 225 | 55 | 45 | 48 | 111 | 34 | 100 |
Prescribed (Others) | 13 | 110 | 84 | 583 | 23 | 50 | 51 | 148 |
Not Prescribed | 0 | 7 | 195 | 1326 | 6 | 5 | 182 | 404 |
Total (3831) | 39 | 342 | 334 | 1954 | 77 | 166 | 267 | 652 |
Number of ARI visits for which antibiotics were given (“Prescribed” rows) or withheld (“Not Prescribed” row) either in accordance with (“Warranted” columns) or against guideline recommendations (“Unwarranted” columns). Columns further separate the visits by 1) study site (Intervention vs. Control Site); 2) time periods (pre-intervention year (“2002”) vs. post-intervention years (“2003–6”)); and 3) by whether or not the antibiotics prescribed were those targeted by the intervention (“Targeted” vs. “Other”).
Comparison of antibiotic utilization for ARI between the time periods before (dark bars) or after (light bars) introduction of the CDSS. Y-axis represents proportions of ARI visit where antibiotics were prescribed. For comparisons involving antibiotics targeted by the CDSS (dark and light bar pairs over the word “Targeted”), proportions are given by VTargeted/(VTargeted+VNoAntibiotic), where VTargeted is the number of visits where targeted antibiotics were prescribed and VNoAntibiotic the number of visits not issued antibiotics. For comparisons involving other antibiotics (dark and light bar pairs over the word “Other”), proportions are given by VOther/(VOther+VNoAntibiotic), where VOther is the number of visits where Other antibiotics were prescribed. Upper panel only includes ARI visits where antibiotics were indicated; lower panel only includes those ARI visits where antibiotics were not indicated. Results for the intervention and the control sites are given on the left and right side of the figure, respectively. Note that proportions of visits where antibiotics were prescribed did not change pre vs. post-intervention, except for a decrease in Targeted antibiotics use at the intervention site.
The majority of ARI visits did not include documentation supporting the use of antibiotics (n = 3207 or 83.7% of total ARI visits,
The proportion of total ARI visits where antibiotics use was congruent with the guidelines increased from the pre- to the post-intervention periods at the intervention site (from 0.63 to 0.72, p = 0.0001), but was unchanged at the control site (from 0.74 to 0.69, p = 0.10). At the intervention site, congruence increased in the first post-intervention year (0.72 (95% CI = (0.68, 0.75) in 2003) and remained stable afterwards (0.73 (95% CI = (0.69, 0.77) in 2004, 0.72 (95% CI = (0.69, 0.75) in 2005, and 0.73 (95% CI = (0.69, 0.78) in 2006). Prevalence ratios of antibiotics prescribing congruence were obtained by site and by year through adjusted multivariable logistic regression models. Adjusted multivariable difference-in-difference models between the two study sites, post- vs. pre-intervention periods, revealed a relative risk (RR) of a congruent prescription to be 1.24 (95% CI = (1.110, 1.391)), in favor of the intervention site. The impact of the intervention was found to be greatest for the targeted antibiotics (RR = 2.57; 95% CI = (1.865, 3.540)). Prescribing congruence for antibiotics not targeted by the intervention was unchanged (adjusted RR = 1.18; 95% CI = (0.691, 2.011)). No change was detected in congruence when no antibiotics were prescribed (adjusted RR = 0.99; 95% CI = (0.990, 1.00)). Favorable adjusted relative risks persisted for the sub-group of patients without pneumonia (n = 3294, RR = 1.27, 95% CI = (1.112, 1.457)) and in patients with acute bronchitis as their only ARI diagnosis (n = 1233, RR = 1.32; 95% CI = (1.042, 1.678)).
In this work, a CDSS interposed treatment guidelines at the time of electronic order entry for antibiotics frequently used for outpatients with ARI. We report that the indicated use of the two antibiotics targeted by the CDSS, azithromycin and gatifloxacin, remained undiminished, but that their unnecessary use for ARI was curtailed for a 4-year period. This outcome was not observed for antibiotics not subject to the CDSS at the intervention and at the control sites.
The strengths of our study include the long duration of the intervention, the large sample size, and explicit case definitions and treatment criteria. Statistical comparisons could be made not only across the pre vs. post intervention time period, but also between the targeted and non-targeted antibiotics. Because pre- and post-intervention data was available from the intervention and the control site, we could also use a quasi-experimental difference-in differences approach to control for factors other than the CDSS that could be contributing to time-dependent changes in congruence to ARI antibiotics guidelines. In absolute terms, the overall 9.5% post-intervention decline in unwarranted antibiotic use for ARI was consistent with the 9.7% median reduction observed in 30 conventional intervention trials reviewed by Ranji et al.
From a safety and tolerability standpoint, providers must be allowed to override the recommendations of a CDSS. This design requirement could have allowed providers to bypass the chief aim of the intervention, which was to convince them not to prescribe antibiotics unnecessarily for ARI. In a first scenario, they could have redirected ingrained misutilization to antibiotics not subject to the CDSS. Had this been the only effect of the intervention, we estimate that the proportion of ARI visits where antibiotics were not warranted but where agents other than azithromycin and gatifloxacin were prescribed should have risen above 50% in the post-intervention period. Because this proportion remained unchanged at 30%, our data argue that the CDSS did not merely shunt misutilization toward alternative, unrestricted drugs. In a second scenario, providers could have assigned the diagnosis of “pneumonia” more liberally, thereby seemingly justifying antibiotics that, in fact, were not indicated. Had providers used this tactic to justify all unwarranted prescription of the targeted agents, the proportion of all ARI visits where the targeted antibiotics were prescribed would have remained unchanged. In reality, this proportion decreased from 21.7% at baseline to 11.8% post-intervention. Providing further reassurance that outcomes were not due to systematic “gaming” of the process, subgroup analyses that either excluded patients with a pneumonia diagnosis or that included only patients whose sole diagnosis was acute bronchitis yielded findings comparable to those found in the full cohort. Overall, and even though unintended actions such as those outlined in the above scenarios could have occurred more than occasionally, our data suggest that the main effect of the CDSS was to extinguish unneeded prescriptions of the targeted agents.
Many factors could limit the generalizability of our results. The study did not employ a randomized allocation process, leaving it susceptible to well-described biases
The rate at which antibiotics are inappropriately used for ARI remains high but has been decreasing for more than a decade
From a disease-management standpoint, this CDSS intervention stood at a disadvantage because it targeted only a minority of the agents that could be used to treat ARIs. The system could nevertheless have effected large changes in overall guideline-congruent prescribing for ARI if it had fostered the transmission of information from providers to providers. Two lines of reasoning suggest that extra-CDSS educational transmission was not a major outcome of this intervention: 1) congruence gains were attributable to improved utilization of the CDSS-targeted antibiotics only; and 2) those gains were realized in the first post-intervention year and did not further increase afterwards. Thus, prescribing congruence did not exhibit the gradual expansion that would have been expected from an increasing the proportion of providers familiar with the ARI guidelines. Whether or not these outcomes are particular to a teaching institution with rapid housestaff turnover, they serve as a reminder that much more work will be required before we know how to best design, target and integrate prescription-based interventions to optimize the overall management of ARI.
This material is the result of work supported with resources and the use of facilities at the VA Maryland Health Care System and the VA Salt Lake City VA Health Care System (including VA Salt Lake City IDEAS Center). The authors would like to thank Shawn Loftus and Robert Sawyer MD, Brett South, MS and Shuying Shen, MStat for technical help with the institutional databases, the VA Informatics and Computing Infrastructure (VINCI) team for data acquisition and processing; Philip VanCamp, Carol Rudo RPh, Christopher Gallagher RPh, Nimalie Stone MD and Trish M. Perl MD for their help with the CDSS content and with the design and implementation of the CDSS process.