Figures
Abstract
Objectives
The aim of this study was to investigate the long-term effect of a teaching intervention designed to reduce undertriage rates in older ED patients. Further, to test the hypothesis that non-adherence to the Emergency Severity Index (ESI) triage algorithm is associated with undertriage. Additionally, to detect patient related risk factors for undertriage.
Methods
Pre-post-test design. The study sample consisted of all patients aged 65 years or older presenting to the ED of an urban tertiary and primary care center in the study periods. A teaching intervention designed to increase adherence to the triage algorithm. To assess, if the intervention resulted in an increase of factual knowledge, nurses took a test before and immediately after the teaching intervention. Undertriage rates were assessed one year after the intervention and compared to the pre-test period.
Results
In the pre-test group 519 patients were included, and 394 in the post-test-group. Factual knowledge among triage nurses was high already before the teaching intervention. Prevalence of undertriaged patients before (22.5%) and one year after the intervention (24.2%) was not significantly different (χ2 = 0.248, df = 1, p = 0.619). Sex, age, mode of arrival, and type of complaint were not identified as independent risk factors for undertriage. However, undertriage rates increased with advancing age. Adherence to the ESI algorithm is associated with correct triage decisions.
Citation: Grossmann FF, Zumbrunn T, Ciprian S, Stephan F-P, Woy N, Bingisser R, et al. (2014) Undertriage in Older Emergency Department Patients – Tilting against Windmills? PLoS ONE 9(8): e106203. https://doi.org/10.1371/journal.pone.0106203
Editor: María Mar Abad-Grau, University of Granada, Spain
Received: April 22, 2014; Accepted: August 1, 2014; Published: August 25, 2014
Copyright: © 2014 Grossmann et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: The authors confirm that all data underlying the findings are fully available without restriction. Data are available upon request because of an ethical and legal restriction as they include human participants. Data may be requested of http://eknz.ch/, the ethical committee in charge – identifier: 109/11.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Undertriage, an assignment of an inadequately low triage level, increases the patients' risk for health status deterioration while waiting. To date, a threshold for undertriage rates is not generally defined in the literature, but to achieve an undertriage rate of less than 10% is recommended [1]. Older ED patients are a vulnerable patient group and are at risk of undertriage [2], [3]. Reasons for this are poorly understood, but likely are multifactorial. The interpretation of vital signs, for instance, is more difficult in older patients, as they may be normal even in serious disease [4]–[6]. Further, older ED patients often present with non-specific complaints such as weakness [7], which may lead to undertriage [8].
However, when correctly applied, the Emergency Severity Index (ESI) proved to be valid and reliable for all ED patients, also in the German version [1], [3], [9], [10]. When using the ESI as a triage instrument, the two main pitfalls of triage in older ED patients appear to be non-adherence to the algorithm and neglect of high risk situations [3]. Presence of a high risk situation or changes in vital signs determines whether a patient can be left waiting or cannot wait (ESI 3 versus 2). This is of importance as delayed care and a prolonged stay in the ED can lead to adverse events e.g. by delaying transfer of critically ill patients to the intensive care unit [11], [12].
Tackling the phenomenon of undertriage therefore might help to detect older ED patients at risk early in the evaluation process. The objective of this study was to test if a teaching intervention had a sustainable long-term effect on reducing undertriage rates in older patients. Further, we aimed to test the hypothesis that non-adherence to the triage algorithm is associated with undertriage. Additionally, we sought to detect patient related risk factors and reasons for undertriage in older ED patients.
Methods
Study design
We chose a pre-post-test design. Data of a previous evaluation were used for the pre-test analysis [3]. To investigate if our teaching intervention had a sustainable long-term effect, post-test data were collected one year after the teaching intervention. We deliberately chose a one year interval in order to exclude an intervention effect leading to a mere short term improvement immediately after training.
Sample and Setting
The samples consisted of consecutive ED patients aged 65 years or older. For the pre-test, we used data from a previous evaluation, which was conducted between April 6th and 27th in 2009. During that period, 519 patients were included [3]. The teaching intervention occurred in March 2010. Sampling for the post-test period was conducted between May 1st and May 17th in 2011. In this period, 505 patients were seen in the ED. Assuming a reduction of the proportion of undertriaged patients from an estimated 22.5% in the pre-test data by half in the post-test data, ca. 230 patients per group (pre- and post-test) would be needed to show a statistically significant difference in proportions with a power of 90% at a significance level of 0.05.
The study was conducted at the ED of the University Hospital Basel, Switzerland, an urban tertiary care center, with an annual census of 42′000 ED visits. Adult patients of all specialties except pediatrics, gynecology and obstetrics, ophthalmology, and odontology are treated in our ED. All patients were triaged using the German translation of the ESI [9]. All triage nurses were either certified in emergency nursing or had longstanding experience in emergency nursing. They trained according to the recommendations of the ESI implementation handbook [1], which is a basic, formal four hour training. In our institution, the triage decisions of newly trained nurses are supervised during the first day of triage. Four education sessions per year including discussion of case scenarios are mandatory for triage nurses.
Selection of Patients
The study sample consisted of all patients aged 65 years or older presenting to the ED in the study periods. Enrolment was done consecutively. Patients where no triage level was documented were excluded from the analysis.
Intervention
To increase adherence to the triage algorithm, we designed a teaching intervention to facilitate a correct triage level assignment in older ED patients. The intervention consisted of a 1 hour lecture mandatory for all triage nurses. The content of the training session was based on the results of our former study, specifically addressing the pitfalls [3] (neglect of high risk situations, inadequate interpretation of vital signs). Real, exemplary cases of undertriage were presented and extensively discussed with 2 triage experts (FFG and CHN).
Methods of Measurement
Effect of the teaching intervention on triage nurses' factual knowledge.
To test if the teaching session resulted in an increased knowledge on the pitfalls of triage of older patients, participants took a test before and immediately after the teaching session. The test consisted of 6 case scenarios on which the participants had to assign a triage level as well as a multiple choice question on the definition of live saving interventions. The case scenarios were identical before and after the teaching sessions. The participants were blinded to the test results until the second test was completed.
Determination of a triage level.
The ESI level was determined by triage nurses following the four decision points A to D of the ESI triage algorithm [1]. ESI level 1 is assigned to patients who need an immediate life-saving intervention (decision point A). ESI level 2 represents patients who should not wait because of a high-risk situation, a new onset of confusion, lethargy, disorientation, or severe pain or distress (decision point B). ESI levels 3, 4, and 5 are assigned to patients who need more than one, one, or no resources, respectively (decision point C). Before assigning ESI level 3 to a patient, vital signs must be assessed (decision point D). If they are outside defined limits (heart rate >100/min, respiratory rate >20/min, or oxygen saturation <92 percent), the triage nurse must consider assigning ESI level 2 [1].
Inadequate triage.
Inadequate triage was defined as incongruence between the triage nurse's ESI assignment and the consensual ESI level retrospectively assigned by two triage experts. Undertriage was defined as cases where the triage nurse's ESI level indicated lower acuity than the experts' ESI level. Overtriage was defined accordingly.
Reasons for inadequate triage.
Reasons for under- and overtriage were recorded and classified according to the decision points of the ESI algorithm A–D. We coded the reason for undertriage as unknown in cases where the triage decision could not be reconstructed based on the triage nurses' notes. In these four cases the experts' triage designation was based on the available data.
Adherence to the triage algorithm.
The triage nurse must consider assigning an ESI level 2 if vital signs are in the danger zone (decision point D). Thus, one possibility to measure adherence to the triage algorithm retrospectively is to test whether vital signs are correctly assessed at decision point D. In cases where the algorithm required no vital sign assessment (decision points A, B, and C), adherence could not be measured. Consequently, we defined the triage nurse as being non-adherent if no vital signs were documented at decision point D, as adherent if relevant vital signs were assessed and as fully adherent if a full set of vital signs was documented including heart rate, blood pressure, temperature, oxygen saturation, and respiratory rate.
Covariates.
Age, gender, and time of arrival were retrieved from the hospital patient database; types of complaints (defined as non-specific, specific, or trauma following a previously published framework [13]) and mode of arrival (walk in or direct ED boarding) were abstracted from patient charts.
Data collection and processing
Pre-test data were collected in the period from April 6th to April 27th 2009 [3]. The intervention took place in March 2010. For the post test period (from May 1st until May 17th 2011), data were retrieved from patient records. A chart abstractor (SC) presented the triage nurses' notes to two experts (CHN, FFG), who reviewed the triage nurses' notes or handover protocols of direct boarders and independently assigned an ESI level according to the decision points of the ESI algorithm. Assessors were blinded to the ESI level assigned by the triage nurse and to all patient outcomes. This approximates decision-making in a real triage environment, where no additional information beside the patient's history is available. If experts disagreed on an ESI assignment, a consensus was reached by discussing the case. In a second step, charts were analyzed by both triage experts in order to determine reasons for inadequate triage, using a standardized abstraction form.
For data management and analyses, R 3.0.2 [14] and SPSS 21.0 [15] were used. The study was approved by the local ethics committee “Ethikkommission Nordwest- und Zentralschweiz” (EKNZ, identifier No 109/11). Written informed consent was not obtained, as the study was classified as a quality control measure. All data were anonymized and de-identified before analysis.
Primary data analysis
To evaluate if the teaching intervention had an immediate effect on factual knowledge, a paired Wilcoxon signed ranks test was used comparing the results of the test before and immediately after the teaching session. To test the long-term effect of our teaching intervention on undertriage, we compared prevalence rates for undertriage in the pre- with the post-test period. We used the Pearson's χ2 test with Yate's continuity correction. The same test was used to compare undertriage rate per ESI level between pre- and post-test. Risk factor analysis was performed by fitting simple logistic regression models for the response “undertriage” and a series of candidate risk factors in the pre- and post-test groups as well as in the combined data set. To illustrate age dependency of undertriage, probability density estimates of the age of undertriaged patients relative to the probability density estimates of age of all patients were calculated. To test whether adherence has an effect on correct triage, logistic regression models for the response “correct triage” were fit for the case that some vital signs were assessed at triage (adherence) and for the case that vital signs including respiratory rate were assessed (full adherence).
Results
Sample characteristics
In the pre-test group, 519 patients were included [3]. In the post-test period, 511 patients aged 65 years or older were treated in the ED. Of these, 117 patients had to be excluded. Of these 3 patients were referred from other hospitals, 6 records were not available, and 108 patients had no ESI level assigned (of which 65 were direct-to-bed patients, and 43 for unknown reason),; In the final analysis, 394 patients were included. The time period needed to include approximately 500 consecutive older ED patients was shorter in the post-test-period, reflecting an increase in ED census.
Characteristics of both groups are shown in table 1: The proportion of ESI levels differs significantly between the pre- and the post-test period for ESI level 1, 2, and 3 (Pearson's χ2 test with Yate's continuity correction) with ESI 1 and 2 being more and ESI 3 being less prevalent in the post-test period (figure 1).
The p values are from the corresponding Pearson's χ2 test with Yate's continuity correction.
Patients were triaged by 16 different nurses in the pre-test period, and 17 nurses in the post-test period respectively. Due to shift planning, 3 nurses, who triaged in the pre-test period were not on duty in the post-test period, and four nurses were new triage team members at the time the intervention took place. All triage nurses who participated in the post-test period had received a teaching intervention. Of these 17 nurses, 11 nurses participated in the teaching session as described, and 6 were trained individually.
Prevalence of and reasons for undertriage
The difference between the prevalence of undertriaged patients before (22.5%) and after the intervention (24.2%) was not significant (χ2 = 0.227, df = 1, p = 0.634. The proportions of undertriage were compared per ESI level. None of the five comparisons resulted in a significant difference (ESI 1: no comparison possible because there were no undertriaged patients, ESI 2: p = 1.000, ESI 3: p = 0.198, ESI 4: p = 0.281, ESI 5: p = 0.333). Reasons for undertriage are shown in table 2.
Effect of the teaching intervention on factual knowledge
Of the 11 eleven triage nurses, who participated in the teaching session, ten took the test before and after the lecture. One nurse had to leave the session before the second test. The six nurses who were trained individually did not take the test. An increase of factual knowledge could not be detected; the mean results were 4.9 points before and 5.3 points (compared to a maximum of 6) after the lecture. The difference for the 10 available pairs was not significant (Wilcoxon signed ranks test, with continuity correction, V = 16.5, p = 0.492).
Interrater reliability of expert ratings
Interrater reliability testing resulted in high agreement between the two experts: Raw agreement was 0.893 (95% CI 0.863–0.924), Cohen's weighted kappa was 0.908 (95% CI 0.877–0.941), and Spearman's rho was 0.908 (0.875–0.942).
Risk factors for undertriage
Sex, age, mode of arrival, and type of complaint considered individually had no significant effect on undertriage (logistic regression models, Table 3). However, when looking at the relative probability density estimates (probability density estimates of age of undertriaged patients relative to probability density estimates of age of all patients) undertriage seems to increase with age in the post treatment group (Figure 2).
The proportions are given as ratios of the kernel probability density estimate of age of patients with undertriage and the kernel probability density estimate of age of all patients.
Adherence
Of the 176 patients with ESI level 3 assigned by the triage nurses, 156 patients (88.6%) had relevant vital signs assessed at triage, reflecting adherence according to our definition. Full adherence is given when a full set of vitals was assessed, which was only the case in 28 patients (15.9%). Assessment of respiratory rate was significantly associated with correct triage decisions (Table 4).
Discussion
The rates of undertriage were stable over time in our study. The teaching intervention geared towards adherence to the ESI algorithm did not result in a sustainable decrease of undertriage rates one year after a teaching intervention. Factual knowledge was consistently high before and immediately after the intervention suggesting that undertriage is not merely a matter of factual knowledge. Independent risk factors for undertriage could not be detected, but a trend towards an increase of undertriage rates with advancing age was observed. Full adherence to the triage algorithm was rare, but if respiratory rate was assessed, this was associated with correct triage decisions.
Undertriage affects patient safety, but it is poorly understood and rarely studied in the clinical ED setting [2], [3], [16]. Despite the paucity of research in this field, geriatric-specific enhancements to the ESI triage algorithm have been suggested [17]. Other studies found that the ESI algorithm itself may be inaccurate in identifying life threatening conditions in older patients [16].
We are not aware of any study that tested interventions dedicated to reduce undertriage (in older ED patients). Our intervention focussed on adherence to the existing ESI triage algorithm. We showed that adherence in terms of measuring respiratory rate is associated with correct triage decisions. However, respiratory rate was only rarely assessed, although its measurement is mandatory in the algorithm at decision point D. Vital sign assessment helps identifying those patients who are in need of more resources and cannot wait. As vital signs are predictive of patient outcomes, complete vital sign assessment at decision point D is of importance [18]. Inconsequent measurement of respiratory rates in our study is a finding that is in line with the literature [19]. Since correct assessment of respiratory rate takes 60 seconds, triage nurses might decide to omit respiratory rate assessment in favour to a more rapid workflow. In addition, the respiratory rate is the only vital sign that cannot be measured with devices and might therefore be not performed. This is despite the fact that respiratory rate has additional diagnostic value in a variety of clinical situations and is incorporated in clinical risk scores and guidelines [20]–[22].
Another issue determining whether a patient can wait or cannot wait is the presence of a high risk situation. High risk situations can be identified by certain complaints, red flag signs, or the medical history. Given the high level of factual knowledge as tested with the present study, one might speculate that triage nurses consciously, against their better knowledge, chose to deliberately deviate from the triage protocol. Especially when triage nurses feel as gate-keeper of a crowded ED, they are reluctant to assign ESI level 2 to patients because immediate placement in the ED treatment area is required [1]. Therefore, environmental factors like ED crowding may contribute to higher rates of undertriage which is supported by a qualitative study on triage decision making [23].
Further, triage decision making in our study might have been biased. Although age did not reach statistical significance in the risk factor analysis – possibly due to small sample size in the very old age groups – there seems to be a trend towards undertriage with increasing age.
We learned from our study that more efforts than a single teaching intervention are needed to overcome undertriage in older ED patients. As triage decisions are made by a heterogeneous collective of triage nurses, individual triage performance should be monitored. Low performers may profit from interventions tailored towards their needs.
To explain the stability of undertriage pattern over time, several contributing factors can be considered. Obviously, factual knowledge about the ESI triage algorithm does not necessarily translate into triage nurses' adherence in a real triage environment. We hypothesize that awareness of the special care needs of older ED patients, of potential age-bias, and of environmental factors might have a positive impact on adherence to the triage algorithm. Further research, such as qualitative studies might help to understand triage decision making in this vulnerable patient group. To confirm that undertriage rates increase with advancing age, quantitative studies with larger sample sizes in the oldest age groups should be performed.
Limitations
The quasi experimental pre-post-test design of this single center study is subject to unmeasured differences between the two periods. A further limitation is the retrospective triage level assignment by the experts. However, retrospective chart review is an accepted method to determine triage quality [1].
In a group of patients, no triage level was documented leading to possible selection bias, which is reduced by consecutive sampling. Additionally, changes in the ED patient population with higher acuity patients and higher overall ED census in the post-test period might have influenced undertriage rates.
Lastly, the list of potential risk factors for undertriage may have been incomplete. We did not assess for association of undertriage with other potential risk factors such as dementia, residence in a nursing home, or do not resuscitate (DNR) orders leading to the possibility of confounding and bias.
Conclusion
Misapplication of existing triage criteria is an important reason for undertriage in older patients when applying the ESI. Apart from non-adherence, reasons for undertriage in older ED patients appear to be more complex than anticipated. Improving awareness of additional contributing factors such as age bias might have a positive impact on triage decision-making.
Acknowledgments
The authors would like to thank the triage nurses of our ED, and Philipp Ludäscher, Corinne Schneider, Gertraud Haberkorn, Michael Mühlmeyer for helpful discussions.
Author Contributions
Conceived and designed the experiments: FFG CHN. Analyzed the data: FFG CHN TZ SC FPS NW. Contributed to the writing of the manuscript: FFG RB CHN.
References
- 1.
Gilboy N, Tanabe P, Travers D, Rosenau A (2011) Emergency Severity Index (ESI): A Triage Tool for Emergency Department Care, Version 4. Imlementation Handbook 2012 Edition. Rockville: Agency for Healthcare Research ans Quality.
- 2. Rutschmann OT, Chevalley T, Zumwald C, Luthy C, Vermeulen B, et al. (2005) Pitfalls in the emergency department triage of frail elderly patients without specific complaints. Swiss Med Wkly 135: 145–150.
- 3. Grossmann FF, Zumbrunn T, Frauchiger A, Delport K, Bingisser R, et al. (2012) At Risk of Undertriage? Testing the Performance and Accuracy of the Emergency Severity Index in Elderly ED Patients. Ann Emerg Med 60: 317–325.
- 4. Lehmann R, Beekley A, Casey L, Salim A, Martin M (2009) The impact of advanced age on trauma triage decisions and outcomes: a statewide analysis. Am J Surg 197: 571–574 discussion 574–575.
- 5. Lim WS, Macfarlane JT (2001) Defining prognostic factors in the elderly with community acquired pneumonia: a case controlled study of patients aged > or = 75 yrs. Eur Respir J 17: 200–205.
- 6. Pines JM, Prosser JM, Everett WW, Goyal M (2006) Predictive values of triage temperature and pulse for antibiotic administration and hospital admission in elderly patients with potential infection. Am J Emerg Med 24: 679–683.
- 7. Nickel CH, Nemec M, Bingisser R (2009) Weakness as presenting symptom in the emergency department. Swiss Med Wkly 139: 271–272.
- 8. Ruger JP, Lewis LM, Richter CJ (2007) Identifying high-risk patients for triage and resource allocation in the ED. Am J Emerg Med. 25: 794–798.
- 9. Grossmann FF, Nickel CH, Christ M, Schneider K, Spirig R, et al. (2011) Transporting clinical tools to new settings: cultural adaptation and validation of the emergency severity index in german. Ann Emerg Med 57: 257–264.
- 10. Baumann MR, Strout TD (2007) Triage of geriatric patients in the emergency department: validity and survival with the Emergency Severity Index. Ann Emerg Med 49: 234–240.
- 11. Parkhe M, Myles PS, Leach DS, Maclean AV (2002) Outcome of emergency department patients with delayed admission to an intensive care unit. Emerg Med (Fremantle) 14: 50–57.
- 12. Yurkova I, Wolf L (2011) Under-triage as a significant factor affecting transfer time between the emergency department and the intensive care unit. J Emerg Nurs 37: 491–496.
- 13. Nemec M, Koller MT, Nickel CH, Maile S, Winterhalder C, et al. (2010) Patients presenting to the emergency department with non-specific complaints: the Basel Non-specific Complaints (BANC) study. Acad Emerg Med 17: 284–292.
- 14.
R Core Team (2013) R: A Language and Environment for Statistical Computing. 3.0.2 ed. Vienna, Austria: R Foundation for Statistical Computing.
- 15.
IBM (2012) IBM SPSS Statistics 21.0.
- 16. Platts-Mills TF, Travers D, Biese K, McCall B, Kizer S, et al. (2010) Accuracy of the Emergency Severity Index triage instrument for identifying elder emergency department patients receiving an immediate life-saving intervention. Acad Emerg Med 17: 238–243.
- 17. Tucker G, Clark NK, Abraham I (2013) Enhancing ED triage to accommodate the special needs of geriatric patients. J Emerg Nurs 39: 309–314.
- 18. Prytherch DR, Smith GB, Schmidt PE, Featherstone PI (2010) ViEWS – Towards a national early warning score for detecting adult inpatient deterioration. Resuscitation 81: 932–937.
- 19. Hogan J (2006) Why don't nurses monitor the respiratory rates of patients? Br J Nurs 15: 489–492.
- 20. Fine MJ, Auble TE, Yealy DM, Hanusa BH, Weissfeld LA, et al. (1997) A prediction rule to identify low-risk patients with community-acquired pneumonia. N Engl J Med 336: 243–250.
- 21. Egermayer P, Town GI, Turner JG, Heaton DC, Mee AL, et al. (1998) Usefulness of D-dimer, blood gas, and respiratory rate measurements for excluding pulmonary embolism. Thorax 53: 830–834.
- 22. Subbe CP, Davies RG, Williams E, Rutherford P, Gemmell L (2003) Effect of introducing the Modified Early Warning score on clinical outcomes, cardio-pulmonary arrests and intensive care utilisation in acute medical admissions. Anaesthesia 58: 797–802.
- 23. Andersson AK, Omberg M, Svedlund M (2006) Triage in the emergency department – a qualitative study of the factors which nurses consider when making decisions. Nurs Crit Care 11: 136–145.