Figures
Abstract
Purpose
To develop a clinical staging system based on the PIRO concept (Predisposition, Infection, Response and Organ dysfunction) for hospitalized patients with infection.
Methods
One year prospective cohort study of all hospitalized patients with infection (n = 1035), admitted into a large tertiary care, university hospital. Variables associated with hospital mortality were selected using logistic regressions. Based on the regression coefficients, a score for each PIRO component was developed and a classification tree was used to stratify patients into four stages of increased risk of hospital mortality. The final clinical staging system was then validated using an independent cohort (n = 186).
Results
Factors significantly associated with hospital mortality were • for Predisposition: age, sex, previous antibiotic therapy, chronic hepatic disease, chronic hematologic disease, cancer, atherosclerosis and a Karnofsky index<70; • for Insult/Infection: type of infection • for Response: abnormal temperature, tachypnea, hyperglycemia and severity of infection and • for Organ dysfunction: hypotension and SOFA score≥1. The area under the ROC curve (CI95%) for the combined PIRO model as a predictor for mortality was 0.85 (0.82–0.88). Based on the scores for each of the PIRO components and on the cut-offs estimated from the classification tree, patients were stratified into four stages of increased mortality rates: stage I: ≤5%, stage II: 6–20%, stage III: 21–50% and stage IV: >50%. Finally, this new clinical staging system was studied in a validation cohort, which provided similar results (0%, 9%, 31% and 67%, in each stage, respectively).
Conclusions
Based on the PIRO concept, a new clinical staging system was developed for hospitalized patients with infection, allowing stratification into four stages of increased mortality, using the different scores obtained in Predisposition, Response, Infection and Organ dysfunction. The proposed system will likely help to define inclusion criteria in clinical trials as well as tailoring individual management plans for patients with infection.
Citation: Cardoso T, Teixeira-Pinto A, Rodrigues PP, Aragão I, Costa-Pereira A, Sarmento AE (2013) Predisposition, Insult/Infection, Response and Organ Dysfunction (PIRO): A Pilot Clinical Staging System for Hospital Mortality in Patients with Infection. PLoS ONE 8(7): e70806. https://doi.org/10.1371/journal.pone.0070806
Editor: Cordula M. Stover, University of Leicester, United Kingdom
Received: April 30, 2013; Accepted: June 21, 2013; Published: July 24, 2013
Copyright: © 2013 Cardoso et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This research was supported by an unrestricted grant from ASSUCIP - Associação de Apoio à Unidade de Cuidados Intensivos Polivalente, Hospital de Santo António, Porto, Portugal (Intensive Care Unit Support Association). Teresa Cardoso is funded by a PhD research grant from the Teaching and Research Department (Departamento de Formação, Ensino e Investigação) of Oporto Hospital Centre. Teixeira-Pinto is funded by a clinical research grant from the Science and Technology Foundation (PIC/IC/83312/2007). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
Introduction
According to the 2012 World Health Organization report, infections are among the top three leading causes of death worldwide [1]. Developing new therapies for sepsis has been particularly challenging and the successive failures have been attributed to the inclusion of a very heterogeneous group of patients.
In 2001, the American College of Chest Physicians and the Society of Critical Care Medicine convened a consensus panel, where John Marshall et al [2]. suggested an approach of sepsis similar to the TNM (tumor, nodes and metastases) staging system [3], used for cancer patients both as a prognostic tool and for individualizing therapy. This led to the PIRO concept, which attempts to characterize sepsis across four components: P for ‘Predisposition (P), I for ‘Insult/Infection (I), R for ‘Response (R) and O for ‘Organ dysfunction (O) [2], [4].
This challenging concept took some time before being adopted by the scientific community and was only recently tested in the clinical field. Different approaches have been published, a model development [5], [6] and a scoring system [7]–[9] solely for patients in the intensive care unit (ICU) setting; a score for patients with suspected infection admitted from the Emergency department [10] and finally a study that developed a staging system [11] but solely for patients with severe sepsis and not considering all the originally proposed variables.
The need for a clinical staging system applicable to all hospitalized patients with confirmed infection remained. This would help stratifying patients at risk, assess criteria for specific therapies, predict outcomes and assist in rational enrolment into clinical studies.
The objective of this study was to develop a clinical staging system based on the PIRO concept through a prospective cohort study in a diverse population of patients with infection on hospital admission or throughout their hospital stay. The derived clinical staging system is validated in an independent cohort.
Patients and Methods
A prospective cohort study was conducted in a 600-bed tertiary care university hospital over a 1-year period (1st June 2008 to 31th May 2009). All adult (age≥18 years) infected patients consecutively admitted to the medical, surgical, nephrology or hematology wards of the hospital, or to ICU were included in the first 24 hours of the diagnosis of infection according to the CDC criteria [12]. The inclusion criteria did not include microbiological data in order to obviate delay in study inclusion or retrospective data collection for diagnosis confirmation.
The data collected included all variables defined in the extended sepsis criteria [2] which were grouped according to each PIRO component as follows.
1. Predisposing factors (‘P’) analyzed included: age, sex, season of admission, previous antibiotic therapy (any antibiotic administration with therapeutic intention in the previous month), hospitalization in the previous year, previous instrumentation, Karnofsky index [13] (a value lower than 70 means inability to carry out normal activity or do active work) and premorbid conditions. Chronic morbidities recorded were: immunosuppression (administration of chemotherapy, radiation therapy during 12 months prior to hospital admission or the equivalent of 0.2 mg/Kg/day prednisolone for at least 3 months or 1 mg/Kg/day for a week during 3 months prior to hospital admission or human immunodeficiency virus infection), chronic hepatic disease [14], chronic heart failure [14], chronic respiratory disease [14], hematologic disease [15], cancer [15], chronic renal failure (if there was need for chronic renal support or a history of chronic renal insufficiency with a serum creatinine level over 2 mg/dl), diabetes mellitus (if insulin therapy or oral anti-diabetic drugs were required before the infection) and/or atherosclerosis (if there was a previous history of transient ischemic attack, stroke, angina, myocardial infarction or peripheral arterial disease).
2. Insult/Infection (‘I’) was characterized by: type of infection, categorized as either community-acquired (CAI), if the infection was detected within 48 hours of hospital admission in patients who did not fulfill the criteria for a healthcare-associated infection; healthcare associated (HCAI - using the same criteria that Deborah Friedman used for healthcare associated bloodstream infections regardless of the involved focus of infection) [16] or hospital-acquired (HAI) [12]; focus of infection (categorized as respiratory [12], urinary [12], intra-abdominal [12], or others); microbiology documentation of infection; presence of bacteremia (primary or secondary) [17] and pathogen identification, classified by category (Gram negative, Gram positive, fungus or poly-microbial).
3. Host Response variables (‘R’) included: abnormal temperature (fever or hypothermia), tachypnea, tachycardia, abnormal white blood cells count (leukocytosis, leucopenia), altered consciousness, hyperglycemia in the absence of diabetes, peripheral edema, high serum C-reactive protein and severity of infection as defined in the 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference (infection, sepsis, severe sepsis or septic shock at presentation [2]).
4. Organ dysfunction (‘O’) was assessed by the following variables: hypoxemia, hypotension, high serum lactate, renal dysfunction, high bilirubin, low platelet count, ileus, coagulopathy and total SOFA score [18].
A second cohort was established to validate the proposed clinical staging system. Data for the validation cohort were retrospectively collected and included all patients admitted to the same wards between 1st December 2011 and 31st January 2012 as the derivation cohort, using the same inclusion criteria.
The primary outcome of interest was on in-hospital mortality. All patients had complete follow-up until hospital discharge in both cohorts.
This study was approved by the institutional review board (which includes the Ethics for Health Committee) of Hospital de Santo António, Oporto Hospital Centre, Portugal, and informed consent from the participants was waived due to its purely observational nature.
Statistical Analysis
Continuous variables were described as means and standard deviations (SD) or as medians and inter-quartile ranges (IQR) if they showed a skewed distribution. Categorical variables were described with absolute frequencies and percentages. Student T-tests or Mann-Whitney tests were used to compare continuous values between survivors and non-survivors. For categorical variables, these comparisons were performed using Pearson χ2 test.
To build the prediction models for P, I, R and O, variables with marginal association with mortality in the univariate analysis (p value <0.2) were screened for the multivariate analysis. Four separate logistic regression models - one for each component “P”, “I”, “R” and “O” – were built using stepwise selection on the variables screened in the previous phase. Once the models were fitted, four scores for each patient were computed, representing the probability of death predicted by each model. The four scores were combined into a logistic regression referred to as “combined PIRO”.
The results of the multivariable models are expressed as odds ratio (OR) with 95% confidence interval (CI95%) and p-values. The accuracy of the models was assessed by the area under receiver operating characteristics curve (AU-ROC) and calibration was tested using the Hosmer-Lemeshow goodness-of-fit test.
In order to simplify the computation of the scores for each component, the regression coefficients were multiplied by two and rounded to the nearest integer. This simplified scoring system was then used to compare the scores obtained directly from the derivation models with the non-rounded coefficients. The AU-ROCs of the simplified version were identical to the derivation ones.
After obtaining the new scores for each PIRO component, a classification tree was used to define cut-offs for component score and identify profiles of risk of death across the four PIRO components. Each node split decision in the tree was chosen from the possible cut-offs for all components, maximizing the within-node homogeneity according to Gini's coefficient [19] impurity measure, which is known to be closely related to both, the AU-ROC and the Mann-Whitney-U test [20].
A cross table with all possible profiles resulting from the combination of Predisposition, Infection, Response and Organ dysfunction against hospital outcome was built and analyzed to yield the final algorithm. In order to simplify the presentation of results, the profiles were further clustered into four clinical stages according to the risk of death.
The significance level for all tests was defined as p<0.05. Data were analyzed using SPSS, version 18 for Windows (Chicago, IL).
Results
During the study period, a total of 3733 patients were assessed and 1035 (28%) met the inclusion criteria for having infection according to the CDC definitions of infection and hence were included in the study. Mean (SD) age was 65 (20) years and mean SAPS II was 29 (13), overall hospital mortality rate was 13% (table 1). The median hospital length of stay in the derivation cohort was 11 (7–22) days. A microbiological confirmation of infection was available in 68% of patients (56% for CAI, 73% for HCAI and 83% for HAI).The validation cohort included 186 patients that were significantly older, with a mean age of 69 (17) years and a mean SAPS II of 35 (15) (table 1).
In table 2, a detailed description of patients’ characteristics and their association with hospital mortality, according to the four components of PIRO, is shown. In table 3, variables independently associated with hospital mortality according to each component of the PIRO concept are described. Variables retained in the final model for predisposing factors (“P”) included age, gender, previous antibiotic therapy, chronic hepatic disease, chronic hematologic disease, cancer, atherosclerosis and a Karnofsky index <70. For those characterizing infection, (“I”), only the type of infection was retained. Response (“R”) variables included abnormal temperature, tachypnea, hyperglycemia, and the severity of infection. Organ dysfunction (“O”) was characterized by hypotension and a SOFA score ≥1.
The AU-ROC(CI95%) of predicted probabilities for hospital mortality for each PIRO component and the combined PIRO model, in the derivation and in the validation cohorts, are shown in table 4. The combined PIRO model had an AU-ROC of 0.85(0.82–0.88) in the derivation cohort and 0.84 (0.76–0.91) in the validation cohort. The Hosmer and Lemeshow test did not show evidence for lack of fit in all four components or in the combined model, be it in the derivation or in the validation cohorts (p>0.1). Comparing this new PIRO model with SAPS II, it shows a higher discrimination power, with an AU-ROC of 0.85 vs 0.81. The performance was also different according to different settings: the PIRO score performed better in the ward with an AU-ROC of 0.84 vs 0.78 of the SAPS II while in the ICU setting both had a similar performance, AU-ROC = 0.83 (table 4).
Using the rounded regression coefficients for each variable, a weighted clinical classification rule was generated to yield the PIRO scores for each component (table 5). Figure 1 shows the “Classification tree used to define cut-offs for each score and identify clusters of risk of death in the derivation cohort”, allowing patients’ stratification in risk stages for each variable. Predisposition had three stages: P1(0–2 points), P2(3–4 points) and P3(≥5 points). Infection had two stages: I1(0–1 points) and I2(2 points). Response had two stages: R1(0–3 points) and R2(≥4 points). Organ dysfunction had two stages: O1(0 points) and O2(≥1 points).
Each node split decision in the tree was chosen from the possible cut-offs for all components, maximizing the within-node homogeneity.
Increasing stages were associated with an increase in hospital mortality rate, both in the derivation and in the validation cohorts (table 6). The expected mortality was then computed for all possible PIRO combinations defining patients’ risk profiles (Table S1 - Mortality rate and clinical stage according to patients’ PIRO characteristics, in the derivation cohort). Using this table, the profiles were clustered into four stages of increased risk for hospital mortality (Table 7).
Stage I (defined as [P1−2 I1−2 R1 O1] or [P1 I1 R1 O2]) included 436 patients with low or medium predisposition, low response score (without septic shock), and either no organ dysfunction, regardless of place of acquisition, or with organ dysfunction but without hospital-acquired infection. Patients in stage I had a hospital mortality rate of 2% (CI95%, 0.4–3%).
Stage II ([P1 I1 R2 O2], [P1 I2 R1 O2],, [P2 I1−2 R1 O2] or [P3 I1−2 R1 O1]) included 354 patients with a low predisposition, without hospital-acquired infection, but with a high response score and organ dysfunction or low predisposition, with hospital-acquired infection, low response (without septic shock) but with organ dysfunction. This stage also included patients with medium predisposition score with low response (no septic shock) and organ dysfunction or high predisposition with low response and no organ dysfunction. This group of patients had a hospital mortality rate of 11% (CI95%, 8–15%).
The 197 patients in stage III, ([P3 I1–2 R1 O2]) were patients with high predisposition, low response and with organ dysfunction. These patients had a hospital mortality rate of 31% (CI95%, 25–37%).
Stage IV ([P2–3 I1–2 R2 O2]) included 42 patients with a medium or high predisposition score, a high response and also organ dysfunction, regardless of place of acquisition of infection. Their hospital mortality rate was 71% (CI95%, 58–85%).
In the validation cohort, the mortality rate was 0% in stage I (0/52), 9% in stage II (5/54), 31% in stage III (15/49) and 67% in stage IV (6/9) (Table S2 - Mortality rate and clinical stage according to patients PIRO characteristics, in the validation cohort).
In figure 2 different stages of PIRO obtained according to the different combinations of Predisposition (P1, P2, P3), Insult (I1, I2), Response (R1, R2) and Organ dysfunction scores (O1, O2) are shown.
The numbers represent the mortality rate for each state, e.g., the state P1I1R2O2 had a mortality of 19% in the derivation cohort and 14% in the derivation cohort. Only states with more than 5 patients were considered. The dashes (−) indicate that there was not enough patients in the respective state to evaluate mortality.
Discussion
This study proposes a clinical staging system for patients with infection based on the PIRO concept. It was developed and validated in a large cohort of unselected hospitalized patients with infection because most patients with infection are outside the ICU setting (86% in our study), widening the clinical application of the original concept proposed.
Comparing this new PIRO score with other prognostic scores, namely SAPS II, it performed superiorly (0.85 vs 0.81) with a higher discrimination power, especially in the large sub-group of patients allocated into the ward (0.84 vs 0.78).
Another study [21] has also compared the performance of another PIRO score [11] with APACHE II [14] and MEDS [22] scores in patients admitted into the emergency department with criteria for early goal directed therapy and the severe sepsis resuscitation bundle (that is patients with severe sepsis) and found that for this group of patients, the discrimination power of PIRO and APACHE II (AU-ROC = 0.71) was better than MEDS (AU-ROC = 0.63). The PIRO score proposed by us, presents an even higher discrimination power (AU-ROC = 0.85); however, it should be noted that different populations are included in both studies, but this staging system preformed equally well in more severe populations as it will be discussed later. Four clinical stages of increased risk of hospital mortality were reached, based on “P” characteristics: age, gender, previous antibiotic therapy, chronic hepatic disease, chronic hematologic disease, cancer, atherosclerosis and a Karnofsky index<70; “I”: type of infection; “R”: abnormal temperature, tachypnea, hyperglycemia, and the severity of infection and “O”: hypotension and a SOFA score≥1.
The validation cohort comprised more severe patients, which can perhaps be explained by seasonal variation, as it included patients admitted in the winter, a period coincident to higher occupation rates of hospital beds with more unscheduled admissions. The higher severity of the patients included in the validation cohort is probably the explanation for the higher mortality rate observed. Nonetheless, even in this cohort of more severe patients, the clinical staging system performed equally well, which is a good indication of the generalizability of the model.
Many interventions tested in clinical trials of critically ill patients with severe sepsis have failed to show benefit. One of the potential reasons for this was an inadequate, non-specific selection of patients enrolled in those trials relying mostly on the original systemic inflammatory response syndrome (SIRS) criteria, with equal weight being given to all variables. A clinical staging system derived from a prognostic model, attributing different weights to each co-variable of the four PIRO components, is more likely to better stratify patients and refine inclusion criteria in such trials.
Previous studies have been limited to selected populations or hospital settings [5], [7]–[11]. Lisboa et al [7] developed a score derived from the PIRO concept to predict ICU mortality in patients with ventilator-associated pneumonia. Rello et al [8], performed a similar study focused only on community acquired pneumonia requiring ICU admission. The present study enriches previous findings by including the three most frequent focus of infection which leads to patients requiring hospital care.
Howell et al [10] developed a scoring system on patients admitted to the emergency department with suspected infection. However, they did not follow subsequent information from the hospital course, so patients ultimately found to have a non-infectious diagnosis may have been included. The only inclusion criteria in the present study was the presence of clinical infection, assessed by the CDC definitions [12] which can be done immediately at bed side, thus not delaying patient stratification and its adequate application in the current study was reinforced by the high microbiological documentation rate, minimizing selection bias.
Moreno et al [9] developed a score in ICU patients, from a subset of patients from the SAPS III database, using a modified definition of PIRO (PIR). They excluded patients who died during the first 48 h, which might exclude patients with high response and organ dysfunction scores (like septic shock), thus underweighting these components in their model. Besides including all patients regardless of the severity of infection, the present study evolves further into a clinical staging system.
Rubulotta et al [11] also performed a secondary analysis of two cohorts of ICU patients (PROWESS and PROGRESS databases) and defined a basic phenotypic characterization of patients admitted with severe sepsis. However, in their retrospective study it was not possible to analyze all variables originally proposed [2]. Being prospective and following a rigorous methodology, including a large cohort of unselected hospitalized patients with infection, the present study has reached a pilot clinical staging system that might include all clinical relevant variables.
Classification trees were used to optimize the discrimination ability of the model rather than determine cut-offs heuristically after a logistic regression. Finally an independent validation cohort was used to assess the robustness of the data and over fitting of the derived model.
Although proponents of the PIRO staging system suggested including biomarkers and/or variables reflecting genetic predisposition, these tools are not yet widely or routinely available. Thus, analyses were derived from covariates currently available at bedside, which might help immediate patient management. Data on microbiology documentation and antibiotics appropriateness were also not included, although they could represent major prognostic factors, they are not readily available for early stratification, which is the main goal of the proposed staging system.
The differences in mortality rates between the cohorts can be explained by random variability due to the small sample size of the validation cohort. However, this is a single-center study with a limited number of patients, both in the derivation and in the validation cohorts; nonetheless, it might represent a major step forward in the clinical application of the PIRO concept, expanding its applicability to all hospitalized patients with infection. Further validation in different settings is needed.
In conclusion, it enriches the findings of previous studies by reaching a clinical staging system through its prospective designed with consideration of all proposed variables [2], including patients at various levels of care inside the hospital, with different focus of infection and severity of disease, widening its application to the vast majority of infected patients, with a robust behavior both in the derivation (AU-ROC = 0.85) and in the validation cohort (AU-ROC = 0.84).
At this point, we propose its use mainly after further validation, for early stratification and inclusion in clinical trials. We hope that in the very near future, it can also be useful to tailor individual therapy.
Conclusions
This study proposes a clinical staging system according to the PIRO concept, with stratification of patients according to their risk of death, derived from different scores obtained in Predisposition considering: age, sex, previous antibiotic therapy, chronic hepatic disease, chronic hematologic disease, cancer, atherosclerosis and a Karnofsky index<70; type of infection in Insult/Infection; abnormal temperature, tachypnea, hyperglycemia and severity of infection in Response and hypotension and SOFA score≥1 in Organ dysfunction. It allowed the building of four stages with increased risk of mortality: from stage I [i.e., P1I1R1O1] associated with a low (≤5%) risk of death, to stage IV [i.e., P3I2R2O2], where the risk of mortality is highest (>50%).
Staging infected patients according to the four components of the PIRO system may be a practical and relevant tool in sepsis research. In particular, this new clinical staging system for hospital mortality in patients with infection may prove to become a useful triage tool to design individualized management strategies as well as for refining inclusion criteria in clinical trials.
Supporting Information
Table S1.
Mortality rate and clinical stage according to patients PIRO characteristics, in the derivation cohort. Only states with more than 5 patients were included in the table.
https://doi.org/10.1371/journal.pone.0070806.s001
(DOC)
Table S2.
Mortality rate and clinical stage according to patients PIRO states, in the validation cohort. Only states with more than 5 patients were included in the table.
https://doi.org/10.1371/journal.pone.0070806.s002
(DOC)
Acknowledgments
We are indebted to Dr. Christian Brun-Buisson, MD, Hôpital Henri Mondor, Assistance Publique - Hôpitaux de Paris & Univ. Paris for the critical reading of the manuscript.
Author Contributions
Conceived and designed the experiments: TC IA ACP AS. Analyzed the data: TC ATP PPR. Wrote the paper: TC ATP PPR IA ACP AS. Conceived the study: TC IA ACP AS. Coordinated data collection: TC. Performed the statistical analysis: ATP PR. Drafted the manuscript: TC ATP. Participated in the analysis and interpretation of the data, clinical revising of the manuscript and final approval of the version to be published: TC ATP PPR IA ACP AES.
References
- 1.
WHO (2012) World Health Statistics 2012. In: WHO G, editor. Available: http://apps.who.int/iris/bitstream/10665/44844/1/9789241564441_eng.pdf, Accessed 2010 Jul 1.
- 2. Levy MM, Fink MP, Marshall JC, Abraham E, Angus D, et al. (2003) 2001 SCCM/ESICM/ACCP/ATS/SIS International Sepsis Definitions Conference. Crit Care Med 31: 1250–1256.
- 3. Denoix P (1946) Enquete permanent dans les centres anticancerreaux. Bull Inst Natl Hyg 1: 70–75.
- 4. Angus DC, Burgner D, Wunderink R, Mira JP, Gerlach H, et al. (2003) The PIRO concept: P is for predisposition. Crit Care 7: 248–251.
- 5. Adrie C, Francais A, Alvarez-Gonzalez A, Mounier R, Azoulay E, et al. (2009) Model for predicting short-term mortality of severe sepsis. Crit Care 13: R72.
- 6. Granja C, Povoa P, Lobo C, Teixeira-Pinto A, Carneiro A, et al. (2013) The predisposition, infection, response and organ failure (Piro) sepsis classification system: results of hospital mortality using a novel concept and methodological approach. PLoS One 8: e53885.
- 7. Lisboa T, Diaz E, Sa-Borges M, Socias A, Sole-Violan J, et al. (2008) The ventilator-associated pneumonia PIRO score: a tool for predicting ICU mortality and health-care resources use in ventilator-associated pneumonia. Chest 134: 1208–1216.
- 8. Rello J, Rodriguez A, Lisboa T, Gallego M, Lujan M, et al. (2009) PIRO score for community-acquired pneumonia: a new prediction rule for assessment of severity in intensive care unit patients with community-acquired pneumonia. Crit Care Med 37: 456–462.
- 9. Moreno RP, Metnitz B, Adler L, Hoechtl A, Bauer P, et al. (2008) Sepsis mortality prediction based on predisposition, infection and response. Intensive Care Med 34: 496–504.
- 10. Howell MD, Talmor D, Schuetz P, Hunziker S, Jones AE, et al. (2011) Proof of principle: the predisposition, infection, response, organ failure sepsis staging system. Crit Care Med 39: 322–327.
- 11. Rubulotta F, Marshall JC, Ramsay G, Nelson D, Levy M, et al. (2009) Predisposition, insult/infection, response, and organ dysfunction: A new model for staging severe sepsis. Crit Care Med 37: 1329–1335.
- 12. Garner JS, Jarvis WR, Emori TG, Horan TC, Hughes JM (1988) CDC definitions for nosocomial infections Am J Infect Control. 16: 128–140.
- 13.
Karnofsky DA, Burchenal JH (1949) “The Clinical Evaluation of Chemotherapeutic Agents in Cancer.” In: CM M, editor. Evaluation of Chemotherapeutic Agents Columbia Univ Press. 196.
- 14. Knaus WA, Draper EA, Wagner DP, Zimmerman JE (1985) APACHE II: a severity of disease classification system. Crit Care Med 13: 818–829.
- 15. Le GallJR, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. JAMA 270: 2957–2963.
- 16. Friedman ND, Kaye KS, Stout JE, McGarry SA, Trivette SL, et al. (2002) Health care–associated bloodstream infections in adults: a reason to change the accepted definition of community-acquired infections. Ann Intern Med 137: 791–797.
- 17. Horan TC, Emori TG (1997) Definitions of key terms used in NNIS System. Am J Infect Control 25: 112–116.
- 18. Vincent JL, Moreno R, Takala J, Willatts S, De Mendonca A, et al. (1996) The SOFA (Sepsis-related Organ Failure Assessment) score to describe organ dysfunction/failure. On behalf of the Working Group on Sepsis-Related Problems of the European Society of Intensive Care Medicine. Intensive Care Med 22: 707–710.
- 19.
Gini C, editor (1912) Variabilità e mutabilità (Variability and Mutability). Reprinted in Memorie di metodologica statistica (Ed. Pizetti E, Salvemini, T). Rome: Libreria Eredi Virgilio Veschi (1955) ed. Bologna. 156 p.
- 20. Hand DJ, Till RJ (2001) A Simple Generalisation of the Area Under the ROC Curve for Multiple Class Classification Problems. Machine Learning 45: 171–186.
- 21. Nguyen HB, Van Ginkel C, Batech M, Banta J, Corbett SW (2011) Comparison of Predisposition, Insult/Infection, Response, and Organ dysfunction, Acute Physiology And Chronic Health Evaluation II, and Mortality in Emergency Department Sepsis in patients meeting criteria for early goal-directed therapy and the severe sepsis resuscitation bundle. J Crit Care 27 (4): 362–9.
- 22. Shapiro NI, Wolfe RE, Moore RB, Smith E, Burdick E, et al. (2003) Mortality in Emergency Department Sepsis (MEDS) score: a prospectively derived and validated clinical prediction rule. Crit Care Med 31: 670–675.