Figures
Abstract
Objective
Clinical decision support systems that notify providers of abnormal test results have produced mixed results. We sought to develop, implement, and evaluate the impact of a computer-based clinical alert system intended to improve atrial fibrillation stroke prophylaxis, and identify reasons providers do not implement a guideline-concordant response.
Materials and Methods
We conducted a cohort study with historical controls among patients at a tertiary care hospital. We developed a decision rule to identify newly-diagnosed atrial fibrillation, automatically notify providers, and direct them to online evidence-based management guidelines. We tracked all notifications from December 2009 to February 2010 (notification period) and applied the same decision rule to all patients from December 2008 to February 2009 (control period). Primary outcomes were accuracy of notification (confirmed through chart review) and prescription of warfarin within 30 days.
Results
During the notification period 604 notifications were triggered, of which 268 (44%) were confirmed as newly-diagnosed atrial fibrillation. The notifications not confirmed as newly-diagnosed involved patients with no recent electrocardiogram at our institution. Thirty-four of 125 high-risk patients (27%) received warfarin in the notification period, compared with 34 of 94 (36%) in the control period (odds ratio, 0.66 [95% CI, 0.37–1.17]; p = 0.16). Common reasons to not prescribe warfarin (identified from chart review of 151 patients) included upcoming surgical procedure, choice to use aspirin, and discrepancy between clinical notes and the medication record.
Citation: Cook DA, Enders F, Caraballo PJ, Nishimura RA, Lloyd FJ (2015) An Automated Clinical Alert System for Newly-Diagnosed Atrial Fibrillation. PLoS ONE 10(4): e0122153. https://doi.org/10.1371/journal.pone.0122153
Academic Editor: Kevan L. Hartshorn, Boston University School of Medicine, UNITED STATES
Received: July 2, 2014; Accepted: February 13, 2015; Published: April 7, 2015
Copyright: © 2015 Cook et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited
Data Availability: Data cannot be deposited in a public repository due to privacy concerns. All data are available upon request to Felicity Enders (Enders.Felicity@mayo.edu).
Funding: The authors received no specific funding for this work.
Competing interests: The authors have declared that no competing interests exist.
INTRODUCTION
Ongoing advances in clinical medicine create new opportunities for patient-centered, high-value care, but achieving this potential will require new models for translating evidence into practice. One such approach uses the electronic medical record (EMR) to proactively notify providers of opportunities to optimize care.[1–5] Such clinical alerts have been used to identify potential drug-drug interactions or dose adjustments when prescribing[1] and to notify providers of critical test results[4, 6] or unexpected changes in clinical status.[3] Such interventions have had mixed effects because clinicians often ignore or override the alert[7, 8] or fail to act appropriately;[9] or because of unintended negative consequences such as increased use of follow-up tests without clinical benefit.[10]
A minority of alerts often accounts for the majority of clinically significant benefits, suggesting that focusing on such high-yield alerts (and suppressing low-yield alerts) may enhance the overall value of the alert system.[1, 11] Despite efforts to better understand the factors that influence successful test notification,[1, 11, 12] much remains to be learned about the successful implementation of this technology.[13, 14] One possible solution is to merge the novel finding (e.g., test abnormality) with other clinical information in the EMR to present a more customized notification, which should hypothetically reduce alert fatigue[14] and increase the likelihood of a clinically appropriate response. This study evaluated the effectiveness of such a tailored clinical alert for atrial fibrillation.
Atrial fibrillation is a common clinical problem and is associated with an increased incidence of stroke.[15] Anticoagulation with warfarin significantly reduces stroke incidence,[16] and the benefit is greatest in those with higher risk of stroke. To help clinicians make decisions about anticoagulation, evidence-based guidelines have been developed along with simple rules that classify the patient's risk profile. The "CHADS2" score[17] uses a patient's history of Congestive heart failure, Hypertension, Age>75, Diabetes, and Stroke to create a risk profile; those with a CHADS2 score >2 should receive anticoagulation with warfarin unless clinically contraindicated.[18, 19] Unfortunately, anticoagulation rates remain relatively low,[20–24] meaning that many patients remain at unnecessarily higher risk of stroke.[23, 25] Automated notifications to providers to identify patients with newly-diagnosed atrial fibrillation might help address this gap in quality care. Aside from one clinical trial in progress,[26] we are not aware of any studies of automated result notification of cardiac rhythm disturbances detected on an electrocardiogram (ECG).
The purpose of the present study was to develop, implement, and evaluate the impact of an automated clinical alert system for atrial fibrillation and identify reasons providers do not implement a guideline-concordant response. The intended goal of the clinical alert system was to increase the rate of appropriate prescription of warfarin for stroke prevention.
METHODS
Overview
We conducted a cohort study using historical controls, comparing the prescribing patterns of providers in the first 3 months of the clinical alert against a historical control (the corresponding 3-month period the previous year).
Notification system and clinical rule
MayoExpert is a multifaceted computer-based knowledge resource integrated with the EMR across a multi-site health system.[27] MayoExpert includes answers to frequently-asked questions, institution-approved care process pathways, a directory of topic-specific experts, a portfolio for licensure and credentialing, and integration with a system to notify providers of urgent or unexpected test results. The test notification system was initially developed for two cardiac dysrhythmias diagnosed using the surface ECG: newly-diagnosed atrial fibrillation not on anticoagulation (the focus of the present study) and prolonged QT interval.[28]
In collaboration with board-certified cardiologists we developed a decision rule defining an atrial fibrillation event warranting notification (i.e., previously undiagnosed and not already taking warfarin). In the final rule, a notification would be triggered if all of the following were true:
- Atrial fibrillation is present on the final ECG interpretation;
- Atrial fibrillation is not listed on the patient’s EMR problem list;
- Atrial fibrillation is not present on any ECG in the past five years;
- Recent (within the past 2 months) international normalized ratio (INR) <2, or no recent INR available, or current heart rate >100;
- Not hospitalized on a cardiology or cardiac surgery service.
All ECG interpretations are suggested by the ECG analysis system (MUSE, GE Heathcare), reviewed by an ECG technician, and confirmed by a cardiologist before reaching the EMR as final results.
The decision rule was implemented in the EMR at the Mayo Clinic in Rochester, MN. At the time of this study, Mayo used Centricity Enterprise (GE Healthcare, Seattle, WA) and its integrated expert rule engine (Blaze Advisor, Fair Isaac Corporation, Minneapolis, MN) as the main EMR infrastructure. The decision rule was developed by a multidisciplinary team including clinicians, informaticians, and programmers. Prior to actual use in clinical practice the rule was iteratively pilot tested and revised by "silent" operation, in which the system notified the development team (rather than the actual provider), and patient records were reviewed to confirm or disconfirm the notification as accurate. The notification system became operational in practice on April 30, 2009.
Notifications appear in the recipient’s EMR messaging system Inbox with a priority marker indicating the need for prompt attention (Fig 1). The message notified the physician that, "A semi-urgent incoming ECG result was received for this patient. ECG indicates new onset atrial fibrillation. If you need additional information on the significance of this semi-urgent alert, please click on the following link." This link would open a screen in MayoExpert[27] (Fig 2) containing additional information on atrial fibrillation including the indications for stroke prophylaxis.
Note that the "patient" in this screen shot is not a real person. The link would take them directly to topic-specific information in the MayoExpert knowledge delivery system (see Fig 2).
The link in the EMR message would take providers to this screen.
Participants
Although the notification system monitors patients and alerts providers for both hospitalized and ambulatory patients, we excluded ambulatory patients for pragmatic reasons (the longer time delay before first follow-up, and the use of paper prescribing which limited accurate abstraction of medication changes). We also excluded patients on a cardiology service because these services have independent processes for managing abnormal ECGs. For all notifications triggered in a non-cardiology hospitalized patient in the period December 2009 to February 2010 we extracted clinical information as noted below. We created a historical comparison cohort by identifying patients with newly-diagnosed atrial fibrillation in the corresponding 3-month period one year prior (December 2008—February 2009). To identify this comparison group, we retrospectively applied the decision rule above to all ECGs showing atrial fibrillation during that time period, thus identifying those patients for whom a notification would have been triggered. We selected this time frame to control for possible seasonal variation in patient populations.
Ethics statement
This study was approved by the Mayo Clinic Institutional Review Board. Although it was neither feasible nor required by the review board that we obtain content from individual patients, Mayo Clinic offers patients the opportunity to opt out of all studies using their records, and we adhered to this policy. Information was extracted from medical records using patient unique identifiers without recording names. All data were analyzed anonymously.
Outcomes and instruments
We identified two primary outcome measures—a process measure and a physician behavior. The primary process measure was the number and accuracy of notifications delivered. We used coder-extracted information from the EMR (see below) to verify the accuracy of notification (i.e., was atrial fibrillation newly-diagnosed?). The primary behavior measure was the number of high-risk patients with newly-diagnosed atrial fibrillation eligible for warfarin anticoagulation who received a prescription for warfarin within 30 days of diagnosis. We defined a warfarin-eligible patient as one with no active bleeding and no use of warfarin at hospital admission. We defined a high-risk patient as one with CHADS2≥2, a risk level at which warfarin anticoagulation is recommended by guidelines[18, 19] except in the case of active bleeding. Warfarin was counted as prescribed if warfarin was listed as a current medication in the EMR medication registry, or if warfarin was listed as a medication on the hospital dismissal summary.
Secondary outcomes were use of an appropriate medication (warfarin for any warfarin-eligible patient or aspirin for patients at low risk of stroke [CHADS2<2]), the time delay before prescribing warfarin in warfarin-eligible patients, and the reason for not prescribing warfarin among high-risk warfarin-eligible patients who did not receive a prescription.
Two research assistants experienced in chart abstraction independently reviewed the medical record using a detailed form to determine whether the patient had a prior history of atrial fibrillation and to extract demographic data (emphasizing information to calculate the CHADS2 score [history of congestive heart failure, hypertension, diabetes, and stroke]) and medications prior to hospitalization. Each coder was blinded to the others' codes, but it was impossible to blind coders to the date range of hospitalization. However, to minimize potential bias one coder was kept unaware of the study intent. All discrepancies were resolved by consensus. One author (FJL) reviewed the charts of all patients with newly-diagnosed atrial fibrillation and CHADS2 ≥2 (in both study periods) for whom warfarin was not prescribed to determine the documented reason for not prescribing warfarin.
Statistical analysis
We used the chi-squared test to compare the proportions of patients prescribed indicated medications in the notification and control periods. We used logistic regression to repeat these comparisons while controlling for CHADS2 score and age. We used time-to-event analyses with Cox proportional hazards to compare the time to warfarin prescription between time periods. Other results are reported using measures of central tendency. We used SAS 9.1.3 for all analyses, with a two-sided alpha level of 0.05.
RESULTS
Number and accuracy of notifications
In the three months following implementation of the notification system (December 2009 to February 2010) there were 16,755 ECGs performed on hospitalized patients. During this period the decision rule identified 604 patients as having potentially newly-diagnosed atrial fibrillation (3.6% of all ECGs) and notified the ordering provider via the EMR messaging system. Of these 604, 268 (44%) were ultimately judged in manual review of clinical documentation to be truly newly-diagnosed. The other (not newly-diagnosed) notifications occurred in patients without an ECG showing atrial fibrillation at our medical center in the past five years, but who had been diagnosed with atrial fibrillation at another medical center or whose last ECG-documented episode of atrial fibrillation was more than five years prior.
Comparability of comparison group
In the control time period (before notification) we manually reviewed records and confirmed 226 patients with newly-diagnosed atrial fibrillation (from 17,520 ECGs performed on hospitalized patients). Patient demographics for both notification and control groups are shown in Table 1. Patient groups were similar except for history of congestive heart failure and diabetes, both more common in the control period. Despite this, CHADS2 scores were similar in both groups (mean [SD] 1.59 [1.19] in the notification group vs 1.56 [1.17] in the control group; p = 0.80).
Thirty-five patients were using warfarin at the time of hospital admission and 20 had clinically significant bleeding making them ineligible for anticoagulation (54 patients total); we did not include these patients in subsequent analyses.
Rate and time-to-prescription of anticoagulation
We evaluated the clinical impact of the notification system by comparing the frequency of new warfarin prescription among high-risk warfarin-eligible patients (CHADS2≥2 and no active bleeding) with newly-diagnosed atrial fibrillation in the notification and control period. Among 125 high-risk warfarin-eligible patients in the notification group, 34 (27%) received a prescription for warfarin within 30 days of diagnosis, compared with 34 of 94 (36%) in the control group (odds ratio 0.66 [95% CI, 0.37–1.17]; p = 0.16). We repeated this comparison among all warfarin-eligible patients (both high and low risk of stroke) while controlling for CHADS2 score and age; these analyses again showed no difference between groups (adjusted odds ratio 0.91 [95% CI, 0.60–1.38]; p = 0.65).
We evaluated the time to warfarin prescription using a time-to-event analysis. Among all warfarin-eligible patients (N = 440) the time to warfarin prescription was virtually identical in the notification and control periods (Fig 3; p = 0.74).
This figure depicts the cumulative percentage of warfarin-eligible patients (N = 440) who received a prescription for warfarin in the first 30 days following the initial ECG showing atrial fibrillation. The difference between groups was not statistically significant (p = 0.74).
As a planned secondary outcome, we compared the frequency of an appropriate medication prescription (warfarin for any warfarin-eligible patient, or aspirin for warfarin-eligible low-risk patients [CHADS2<2]). An appropriate medication was prescribed for 109 of 244 (45%) warfarin-eligible patients in the notification period, compared with 85 of 196 (43%) in the control period (odds ratio 1.05 [95% CI, 0.72–1.54]; p = 0.78). Adjusting this analysis for CHADS2 score and age revealed similar results (adjusted odds ratio 1.12 [95% CI, 0.76–1.66]; p = 0.57).
Reasons for non-use of warfarin
To better understand the reasons why warfarin was not prescribed, we reviewed the charts of the 151 high-risk warfarin-eligible patients with newly-diagnosed atrial fibrillation who were not prescribed warfarin (60 in the pre-notification period, and 91 during the notification period). The most common reasons documented by the hospital team are summarized in Table 2. The modal reason, documented in 34 of 151 cases, was a planned surgical procedure, followed by a choice to use aspirin instead of warfarin (N = 27). No reason was documented in 18 cases, and in 16 cases the clinical team documented that the patient was (or should be) receiving warfarin while the EMR medication record did not. Other reasons commonly cited include frequent falls, absence of atrial fibrillation on a subsequent ECG, and high risk of bleeding (see Table 2).
DISCUSSION
We developed, implemented, and evaluated a system to alert providers of newly-diagnosed atrial fibrillation, with the intent of improving guideline-concordant use of warfarin for stroke prophylaxis. This system correctly notified providers of 268 patients with newly-diagnosed atrial fibrillation, yet we found no statistically significant difference from historical controls in the rate of warfarin anticoagulation or the time to prescribing warfarin. Reasons for not using anticoagulation after notification were not always documented, but most often noted temporary withholding of warfarin for a specific procedure, choice to use aspirin instead, and high risk of bleeding.
Limitations and strengths
This study has several limitations. The use of historical rather than randomized controls allows the possibility of systematic differences between the study groups, although we tried to control for seasonal variation and groups were similar in most measured characteristics. We are not aware of any changes in institution procedures that would significantly influence our findings. We excluded ambulatory patients for a number of pragmatic reasons (see Methods), such that these results may not apply to non-hospitalized patients. Only one investigator reviewed charts to identify reasons that warfarin was not prescribed. We did not review all ECGs and patient charts to identify missed cases of newly-diagnoses atrial fibrillation, but during pilot testing we found the decision rule algorithm to be accurate. We did not abstract information on reason for hospitalization. There are also several strengths, including a large sample size, a control group, combined computer and human interpretations of ECGs, analyses stratifying by and adjusting for patient risk, and additional chart abstraction looking for reasons for guideline non-adherence.
Integration with prior work
The computer decision rule notified providers of 268 patients with newly-diagnosed atrial fibrillation after sifting through over 16,000 ECGs and sending 604 notifications. Although notifications proved unnecessary (not newly-diagnosed) in about half the cases, this accuracy is much higher than that of clinical alerts for potential drug-drug interactions[8] and similar to that of systems to notify providers of changes in clinical status.[3, 29] Overall, this suggests relatively acceptable accuracy for this decision rule.
Yet provider responses to notifications were less favorable than we had hoped. Potential explanations for this discrepancy include inappropriate timing or suboptimal design of the notification message (e.g., requiring providers to click to access information); providers' failure to acknowledge the notification (e.g. alert fatigue[14]) or recognize its importance (e.g., failure to recognize a knowledge gap[30] or disagreement with recommendations[5]); and shortcomings in the knowledge resource (e.g., inefficient or non-intuitive organization[31]). These and other factors have been described as "grand challenges" to the implementation and clinical impact of computer-based clinical decision support.[32] Prior research on computer-based decision supports suggests that they often do not work as expected.[33–35] Predictors of success, which remain incompletely clarified, include integration into the clinician's workflow, provision of specific recommendations, inclusion of patient education, and requiring clinicians to justify non-adherence to recommendations.[33, 36, 37] Considering clinical alerts specifically, evidence suggests additional influential factors including providers' awareness of the severity and probability of the potential adverse event, the patient's clinical status and specific risk factors, and the strength of the evidence supporting the recommendation.[5, 11–13] A final challenge is that clinical practice is constantly changing, as illustrated by an updated risk scoring system (the CHA2DS2-VASc) that was first described[38] just as our study ended, and will require a change to our notification decision rule.
Warfarin usage in our sample (about 60% of high-risk patients, not counting those who died or who had an upcoming surgical procedure) was similar to that described in the literature (typically about 40 to 60% of eligible patients).[20, 22–25, 39] In prior studies[25, 39, 40] and in ours, many of the reasons documented for not using warfarin are inappropriate, suggesting knowledge gaps among faculty members regarding the safety of warfarin (e.g., among older patients and those at risk of falls or bleeding[41]). Such gaps between evidence and practice highlight the imperative to change provider behaviors through both notification and education.
An ongoing clinical trial is evaluating a system similar to ours for notifying providers of new-onset atrial fibrillation.[26] Other evaluations of computer-based supports to enhance atrial fibrillation management are limited. One study piloted a computer scoring algorithm to calculate risk scores.[42] Others have used computer-based tools to increase warfarin use among patients in a nursing home,[43] to optimize the timing of follow-up,[44] and to inform provider-patient treatment discussions.[45]
IMPLICATIONS AND CONCLUSIONS
This study highlights the imperative to better understand how to translate a clinical notification into provider action. Since providers are bombarded daily with numerous alerts[4] it may help to further improve the decision rule accuracy, and to tailor the message to more clearly indicate the recommended action. To decrease unnecessary alerts, we are changing the decision rule to exclude patients already recognized as having atrial fibrillation (and who thus do not warrant an alert) by improving linkage with the EMR problem list and medication list. We also plan to adjust the notification message if the duration of atrial fibrillation is unknown (i.e., no prior ECGs available). Automated estimation of stroke risk would further help to tailor notifications.[42] It may reduce alert fatigue to use separate notifications for patients requiring anticoagulation and those with rapid ventricular response. Finally, given the rising use of anticoagulants not monitored with INR, the role of the INR in the decision rule will be revisited.
Yet the accuracy of our system was already relatively good, which highlights the critical role of human factors in translating information into action.[46] Providers now have access to virtually limitless information at the point of care, but continue to struggle in recognizing knowledge gaps, identifying practice updates, and acting on current evidence. In addition to transforming information to actionable knowledge (knowledge creation),[47] effective knowledge translation requires delivery to the right provider at the right time using a format that promotes both immediate action and knowledge retention.[48–50] Computer point-of-care knowledge resources can potentially help with both.[31, 33, 51] The science of knowledge translation is still developing,[52] but preliminary evidence suggests that favorable factors include optimal integration into clinical workflow, provision of specific recommendations, and highlighting the clinical impact of these recommendations along with the strength of the evidence.[13, 33, 36, 37] Notification via an EMR message did not appear to be an effective workflow integration in our study. Further research to iteratively test and optimize clinical alerts is needed, looking first at the impact on provider behaviors and then ensuring that behaviors translate to desired changes in clinical practice.
Acknowledgments
The authors thank Jill Killian and Elysia Jeavons for assistance with statistical analysis; Carol Fitzgerald and Jeanette Christiansen for their assistance in chart review; and Kristi Sorensen and Jane Shellum for their contributions to planning, implementation, and evaluation of the alert system.
Author Contributions
Conceived and designed the experiments: FE RAN FJL. Performed the experiments: FJL. Analyzed the data: DAC FE PJC. Contributed reagents/materials/analysis tools: FE. Wrote the paper: DAC FE PJC RAN FJL.
REFERENCES
- 1. Weingart SN, Simchowitz B, Padolsky H, Isaac T, Seger AC, Massagli M, et al. An empirical model to estimate the potential impact of medication safety alerts on patient safety, health care utilization, and cost in ambulatory care. Arch Intern Med. 2009; 169: 1465–1473. pmid:19752403
- 2.
Shojania KG, Jennings A, Mayhew A, Ramsay CR, Eccles MP, Grimshaw J. The effects of on-screen, point of care computer reminders on processes and outcomes of care. Cochrane Database Syst Rev. 2009: CD001096.
- 3. Kashiouris M, O'Horo JC, Pickering BW, Herasevich V. Diagnostic performance of electronic syndromic surveillance systems in acute care: a system.atic review. Appl Clin Inform. 2013; 4: 212–224. pmid:23874359
- 4. Murphy DR, Reis B, Sittig DF, Singh H. Notifications received by primary care practitioners in electronic health records: a taxonomy and time analysis. Am J Med. 2012; 125: 209 e201–207.
- 5. Weingart SN, Seger AC, Feola N, Heffernan J, Schiff G, Isaac T. Electronic drug interaction alerts in ambulatory care: the value and acceptance of high-value alerts in US medical practices as assessed by an expert clinical panel. Drug Saf. 2011; 34: 587–593. pmid:21663334
- 6. Singh H, Thomas EJ, Sittig DF, Wilson L, Espadas D, Khan MM, et al. Notification of abnormal lab test results in an electronic medical record: do any safety concerns remain? Am J Med. 2010; 123: 238–244. pmid:20193832
- 7. Weingart SN, Toth M, Sands DZ, Aronson MD, Davis RB, Phillips RS. Physicians' decisions to override computerized drug alerts in primary care. Arch Intern Med. 2003; 163: 2625–2631. pmid:14638563
- 8. Isaac T, Weissman JS, Davis RB, Massagli M, Cyrulik A, Sands DZ, et al. Overrides of medication alerts in ambulatory care. Arch Intern Med. 2009; 169: 305–311. pmid:19204222
- 9. Miller AM, Boro MS, Korman NE, Davoren JB. Provider and pharmacist responses to warfarin drug-drug interaction alerts: a study of healthcare downstream of CPOE alerts. J Am Med Inform Assoc. 2011; 18 (Suppl 1): i45–50. pmid:22037888
- 10. Austrian JS, Adelman JS, Reissman SH, Cohen HW, Billett HH. The impact of the heparin-induced thrombocytopenia (HIT) computerized alert on provider behaviors and patient outcomes. J Am Med Inform Assoc. 2011; 18: 783–788. pmid:21712374
- 11. Paterno MD, Maviglia SM, Gorman PN, Seger DL, Yoshida E, Seger AC, et al. Tiering drug-drug interaction alerts by severity increases compliance rates. J Am Med Inform Assoc. 2009; 16: 40–46. pmid:18952941
- 12. Riedmann D, Jung M, Hackl WO, Ammenwerth E. How to improve the delivery of medication alerts within computerized physician order entry systems: an international Delphi study. J Am Med Inform Assoc. 2011; 18: 760–766. pmid:21697293
- 13. Langemeijer MM, Peute LW, Jaspers MW. Impact of alert specifications on clinicians' adherence. Stud Health Technol Inform. 2011; 169: 930–934. pmid:21893882
- 14. Smithburger PL, Buckley MS, Bejian S, Burenheide K, Kane-Gill SL. A critical evaluation of clinical decision support for the detection of drug-drug interactions. Expert Opin Drug Saf. 2011; 10: 871–882. pmid:21542665
- 15. Wolf PA, Abbott RD, Kannel WB. Atrial fibrillation as an independent risk factor for stroke: the Framingham Study. Stroke. 1991; 22: 983–988. pmid:1866765
- 16. Hart RG, Pearce LA, Aguilar MI. Meta-analysis: Antithrombotic Therapy to Prevent Stroke in Patients Who Have Nonvalvular Atrial Fibrillation. Ann Intern Med. 2007; 146: 857–867. pmid:17577005
- 17. Gage BF, Waterman AD, Shannon W, Boechler M, Rich MW, Radford MJ. Validation of clinical classification schemes for predicting stroke: results from the National Registry of Atrial Fibrillation. JAMA. 2001; 285: 2864–2870. pmid:11401607
- 18. Fuster V, Rydén LE, Cannom DS, Crijns HJ, Curtis AB, Ellenbogen KA, et al. ACC/AHA/ESC 2006 Guidelines for the Management of Patients With Atrial Fibrillation—Executive Summary: A Report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the European Society of Cardiology Committee for Practice Guidelines. Circulation. 2006; 114: 700–752.
- 19. Goldstein LB, Bushnell CD, Adams RJ, Appel LJ, Braun LT, Chaturvedi S, et al. Guidelines for the primary prevention of stroke: a guideline for healthcare professionals from the American Heart Association/American Stroke Association. Stroke. 2011; 42: 517–584. pmid:21127304
- 20. Go AS, Hylek EM, Borowsky LH, Phillips KA, Selby JV, Singer DE. Warfarin use among ambulatory patients with nonvalvular atrial fibrillation: the anticoagulation and risk factors in atrial fibrillation (ATRIA) study. Ann Intern Med. 1999; 131: 927–934. pmid:10610643
- 21. Nilsson GH, Bjorholt I. Occurrence and quality of anticoagulant treatment of chronic atrial fibrillation in primary health care in Sweden: a retrospective study on electronic patient records. BMC Clin Pharmacol. 2004; 4: 1. pmid:15028124
- 22. Partington SL, Abid S, Teo K, Oczkowski W, O'Donnell MJ. Pre-admission warfarin use in patients with acute ischemic stroke and atrial fibrillation: The appropriate use and barriers to oral anticoagulant therapy. Thromb Res. 2007; 120: 663–669. pmid:17434577
- 23. Gladstone DJ, Bui E, Fang J, Laupacis A, Lindsay MP, Tu JV, et al. Potentially preventable strokes in high-risk patients with atrial fibrillation who are not adequately anticoagulated. Stroke. 2009; 40: 235–240. pmid:18757287
- 24. Agarwal S, Bennett D, Smith DJ. Predictors of warfarin use in atrial fibrillation patients in the inpatient setting. Am J Cardiovasc Drugs. 2010; 10: 37–48. pmid:20104933
- 25. Ewen E, Zhang Z, Simon TA, Kolm P, Liu X, Weintraub WS. Patterns of warfarin use and subsequent outcomes in atrial fibrillation in primary care practices. Vasc Health Risk Manag. 2012; 8: 587–598. pmid:23112579
- 26. Holt TA, Fitzmaurice DA, Marshall T, Fay M, Qureshi N, Dalton AR, et al. AUtomated Risk Assessment for Stroke in Atrial Fibrillation (AURAS-AF)—an automated software system to promote anticoagulation and reduce stroke risk: study protocol for a cluster randomised controlled trial. Trials. 2013; 14: 385. pmid:24220602
- 27. Cook DA, Sorensen KJ, Nishimura RA, Ommen SR, Lloyd FJ. A comprehensive system to support physician learning at the point of care Acad Med. 2015; 90: 33–39. pmid:25374037
- 28. Haugaa KH, Bos JM, Tarrell RF, Morlan BW, Caraballo PJ, Ackerman MJ. Institution-wide QT alert system identifies patients with a high risk of mortality. Mayo Clin Proc. 2013; 88: 315–325. pmid:23541006
- 29. Nelson JL, Smith BL, Jared JD, Younger JG. Prospective trial of real-time electronic surveillance to expedite early care of severe sepsis. Ann Emerg Med. 2011; 57: 500–504. pmid:21227543
- 30. Eva KW, Regehr G. Knowing when to look it up: a new conception of self-assessment ability. Acad Med. 2007; 82: S81–84. pmid:17895699
- 31. Cook DA, Sorensen KJ, Hersh W, Berger RA, Wilkinson JM. Features of Effective Medical Knowledge Resources to Support Point of Care Learning: A Focus Group Study. PLoS ONE. 2013; 8(11): e80318. pmid:24282535
- 32. Sittig DF, Wright A, Osheroff JA, Middleton B, Teich JM, Ash JS, et al. Grand challenges in clinical decision support. J Biomed Inform. 2008; 41: 387–392. pmid:18029232
- 33.
Lobach D, Sanders GD, Bright TJ, Wong A, Dhurjati R, Bristow E, et al. Enabling health care decisionmaking through clinical decision support and knowledge management. Evidence Report No. 203. Rockville, MD: Agency for Healthcare Research and Quality; 2012. 1–784 p.
- 34. Garg AX, Adhikari NKJ, McDonald H, Rosas-Arellano MP, Devereaux PJ, Beyene J, et al. Effects of Computerized Clinical Decision Support Systems on Practitioner Performance and Patient Outcomes: A Systematic Review. JAMA. 2005; 293: 1223–1238. pmid:15755945
- 35. Koppel R, Metlay JP, Cohen A, Abaluck B, Localio AR, Kimmel SE, et al. Role of Computerized Physician Order Entry Systems in Facilitating Medication Errors. JAMA. 2005; 293: 1197–1203. pmid:15755942
- 36. Kawamoto K, Houlihan CA, Balas EA, Lobach DF. Improving clinical practice using clinical decision support systems: a systematic review of trials to identify features critical to success. BMJ. 2005; 330: 765. pmid:15767266
- 37. Roshanov PS, Fernandes N, Wilczynski JM, Hemens BJ, You JJ, Handler SM, et al. Features of effective computerised clinical decision support systems: meta-regression of 162 randomised trials. BMJ. 2013; 346: f657. pmid:23412440
- 38. Lip GY, Nieuwlaat R, Pisters R, Lane DA, Crijns HJ. Refining clinical risk stratification for predicting stroke and thromboembolism in atrial fibrillation using a novel risk factor-based approach: the Euro heart survey on atrial fibrillation. Chest. 2010; 137: 263–272. pmid:19762550
- 39. Tulner LR, Van Campen JP, Kuper IM, Gijsen GJ, Koks CH, Mac Gillavry MR, et al. Reasons for undertreatment with oral anticoagulants in frail geriatric outpatients with atrial fibrillation: a prospective, descriptive study. Drugs Aging. 2010; 27: 39–50. pmid:20030431
- 40. Rosenman MB, Simon TA, Teal E, McGuire P, Nisi D, Jackson JD. Perceived or actual barriers to warfarin use in atrial fibrillation based on electronic medical records. Am J Ther. 2012; 19: 330–337. pmid:22964558
- 41.
January CT, Wann LS, Alpert JS, Calkins H, Cleveland JC, Cigarroa JE, et al. AHA/ACC/HRS guideline for the management of patients with atrial fibrillation: a report of the American College of Cardiology/American Heart Association Task Force on Practice Guidelines and the Heart Rhythm Society. J Am Coll Cardiol. 2014; Online early (March 2014).
- 42. Grouin C, Deleger L, Rosier A, Temal L, Dameron O, Van Hille P, et al. Automatic computation of CHA2DS2-VASc score: information extraction from clinical texts for thromboembolism risk assessment. AMIA Annu Symp Proc. 2011; 2011: 501–510. pmid:22195104
- 43. Kruger K, Strand L, Geitung JT, Eide GE, Grimsmo A. Can electronic tools help improve nursing home quality? ISRN Nurs. 2011; 2011: 208142. pmid:22013540
- 44. Fihn SD, McDonell MB, Vermes D, Henikoff JG, Martin DC, Callahan CM, et al. A computerized intervention to improve timing of outpatient follow-up: a multicenter randomized trial in patients treated with warfarin. National Consortium of Anticoagulation Clinics. J Gen Intern Med. 1994; 9: 131–139. pmid:8195911
- 45. Fraenkel L, Street RL Jr., Towle V, O'Leary JR, Iannone L, Van Ness PH, et al. A pilot randomized controlled trial of a decision support tool to improve the quality of communication and decision-making in individuals with atrial fibrillation. J Am Geriatr Soc. 2012; 60: 1434–1441. pmid:22861171
- 46. Grimshaw JM, Eccles MP, Lavis JN, Hill SJ, Squires JE. Knowledge translation of research findings. Implement Sci. 2012; 7: 1748–5908.
- 47. Haynes RB, Cotoi C, Holland J, Walters L, Wilczynski N, Jedraszewski D, et al. Second-order peer review of the medical literature for clinical practitioners. JAMA. 2006; 295: 1801–1808. pmid:16622142
- 48. Del Fiol G, Haug PJ, Cimino JJ, Narus SP, Norlin C, Mitchell JA. Effectiveness of Topic-specific Infobuttons: A Randomized Controlled Trial. J Am Med Inform Assoc. 2008; 15: 752–759. pmid:18755999
- 49. Banzi R, Liberati A, Moschetti I, Tagliabue L, Moja L. A review of online evidence-based practice point-of-care information summary providers. J Med Internet Res. 2010; 12(3): e26. pmid:20610379
- 50. Reed DA, West CP, Holmboe ES, Halvorsen AJ, Lipner RS, Jacobs C, et al. Relationship of electronic medical knowledge resource use and practice characteristics with Internal Medicine Maintenance of Certification Examination scores. J Gen Intern Med. 2012; 27: 917–923. pmid:22374410
- 51. Cook DA, Sorensen KJ, Wilkinson JM, Berger RA. Barriers and decisions when answering clinical questions at the point of care: a grounded theory study. JAMA Internal Medicine. 2013; 173: 1962–1969. pmid:23979118
- 52. Curran JA, Grimshaw JM, Hayden JA, Campbell B. Knowledge translation research: the science of moving research into policy and practice. J Contin Educ Health Prof. 2011; 31: 174–180. pmid:21953658