Optimal Subsets of Biomarkers

Multivariate correlation analysis is a general tool for exploring how variables are inter-related. Canonical correlation analysis provides a powerful tool for discovering relationships between two sets of variables. Given two sets of variables, CCA can identify subsets of each set, which when combined as latent variables, produce the maximum correlation between the two sets. In this study, we choose one set of variables to be the sepsis score, and the second set is taken from all possible subsets of the ten biomarkers. CCA can thus generate an ordered list of biomarkers that are most correlated with the sepsis score. See Materials and Methods for details.

Here we discuss the results of applying CCA to select the best combinations of sepsis biomarkers. We first consider the single biomarker with highest correlation to the sepsis score. As shown in Table 1, this biomarker is Bands. If we consider all pairs of biomarkers, Bands and Plt possess the highest correlation with sepsis score. We note that CD64 has the second highest correlation with sepsis score, in the univariate sense, but improves the correlation of Bands to sepsis score less than Plt, which has a lower univariate correlation with sepsis score. This is due to the fact that Bands and CD64 are more correlated than Bands and Plt, and so less information is provided by adding CD64. Hgb enters at even though it has a very weak pairwise correlation with the sepsis score given it also has very weak pairwise correlation with Bands and Plt. The correlation saturates at with the following combination set of biomarkers: Bands, CD64, Segs, WBC, and Plt. The rest of the biomarkers do not provide significant additional information about the sepsis score. The above analysis suggests that these five variables should be included in our sepsis scoring system.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of the Canonical Correlation Analysis and Forward Selection of the biomarkers.

https://doi.org/10.1371/journal.pone.0082700.t001

The Diagnostic Classifier

We seek to construct a decision function from the biomarker data that serves as a hematological scoring system, i.e. a function that maps a sample vector of biomarkers to a positive or negative sepsis diagnosis. Using the biomarkers identified by CCA above, WBC, Plt, Segs, Bands, and CD64, we propose the linear decision function:

From the sparse support vector machine approach described in Materials and Methods, we determined the optimal decision function to be (1)

See Table 2 for the weights, , and their errors, and means and standard deviations of the biomarkers. With this decision function, if the Score is greater than or equal to zero the diagnosis is positive for sepsis, whereas if the Score is less than zero, the diagnosis is healthy or aseptic disease. We note that since the range of values of the biomarkers varies widely, all values of the biomarkers are normalized by subtracting the mean over all cases and then dividing by the standard deviation.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 2. Parameters for the classifier at k = 5.

https://doi.org/10.1371/journal.pone.0082700.t002

The results of applying the classifier in Equation (1) to the full sepsis dataset are shown in Table 3. We calculated the true positive rate (TPR), true negative rate (TNR), positive predictive value (PPV), negative predictive value (NPV), and accuracy (ACC) (defined in Materials and Methods) for these five biomarkers. We emphasize that there are two remaining questions of interest. How good is the classifier? Did we identify the most predictive biomarkers from the original set of ten? We focus on the validation of these biomarkers in the next section.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Table 3. Performance of the classifier at k = 5 for SSVM and LLR.

https://doi.org/10.1371/journal.pone.0082700.t003

Biomarker Validation

In this section, we have two goals. First, we will verify that the number of biomarkers suggested by CCA, , is optimal. Secondly, we seek to provide evidence that the CCA-selected biomarkers are optimal. To do this, we will perform an exhaustive analysis of all possible scoring systems for the ten biomarkers. Clearly this approach is not feasible for large sets of biomarkers, but we exploit the fact that we only have ten to illustrate the power of CCA biomarker selection by constructing all possible SSVM classifiers. We used the accuracy of the resulting decision functions for our validation.

Validation of the Classifier.

For each , we select the -combination set of biomarkers as identified by CCA and shown in Table 1. We construct a decision function for each from to and evaluate several measures of the quality of the scoring system in Fig. 1. We find that each measure begins to saturate near , although one could argue that some slight improvement could be obtained by adding one or two more biomarkers for the given model. (We note that this particular model was not optimized over variations in the parameter .)

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 1. Prediction measures obtained from the (A) Sparse Support Vector Machine and (B) LASSO Logistic Regression methods.

True positive rate (TPR), true negative rate (TNR), positive predictive value (PPV), negative predictive value (NPV), and accuracy (ACC) are shown for each -combination of biomarkers selected.

https://doi.org/10.1371/journal.pone.0082700.g001

Receiver operating characteristic (ROC) curves for true positive versus false positive rate provide additional insight into the determination of the minimal number of biomarkers that provide predictive information about sepsis infection. In Figure 2, we show that the ROC curves become independent of for , and thus is indeed the appropriate number of biomarkers. In the inset to Figure 2, we show the ROC curve for averaged over SSVM models.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 2. Receiver operating characteristic (ROC) curves.

ROC curves of TPR versus FPR for optimal sets of biomarkers where averaged over SSVM models. The shaded region in the inset shows the standard deviation for .

https://doi.org/10.1371/journal.pone.0082700.g002

Validation of the CCA Selected Biomarkers.

We provide further evidence that our CCA biomarker selection was in fact the optimal one by applying SSVM to all possible combinations of biomarkers for each . We show the TPR, TNR, PPV, NPV, and ACC for the top of all possible combinations in Figure 3. It is clear that the CCA-selected biomarkers possess the best statistical measures for each .

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Figure 3. Exhaustive evaluation of statistical measures.

The highest TPR, TNR, PPV, NPV, ACC values when SSVM was applied for all possible combinations of biomarkers (blue circles) from . The solid red circles are the values for models built using the best biomarkers selected by CCA.

https://doi.org/10.1371/journal.pone.0082700.g003

Sepsis Evaluations

The clinical and historical features used to identify patients at risk for sepsis include one or more of the following, as determined by the attending neonatologist [6], [28], [29]: (1) respiratory compromise (e.g. tachypnea, increase in frequency or severity of apnea, or increased ventilator support); (2) cardiovascular compromise (e.g. increased frequency or severity of bradycardic episodes, pallor, decreased perfusion, or hypotension); (3) metabolic changes (e.g. temperature instability, feeding intolerance, glucose instability, or metabolic acidosis); (4) neurological changes (e.g. lethargy, hypotonia, or irritability); and (5) antenatal risk factors (e.g. maternal Group B Streptococcus (GBS) colonization without adequate intrapartum prophylaxis, unknown maternal GBS status, maternal temperature, chorioamnionitis, preterm labor, or prolonged rupture of membranes). After the sepsis evaluation was performed, we utilized the following values derived from CBC to assign a sepsis score [8], [14]: (1) Absolute Neutrophil Count (ANC) or ; (2) Absolute Band Count (ABC) ; (3) Immature to Total neutrophil ratio (IT-ratio) ; and (4) Platelet (Plt) count . Infants who met 2 or more of these laboratory criteria were categorized as having a positive sepsis score. Hemoglobin was measured in the clinical hematology laboratory using a calorimetric method. The hematocrit was calculated after measuring the total red blood cell count (RBC) and the mean corpuscular volume (MCV) of the RBCs. All blood cultures were collected using standard sterile techniques. As per unit protocol, we attempt to obtain 2 blood cultures with a minimum of ml. The BACTEC (Becton Dickinson and Co., Sparks, MD) microbial detection system was used to detect positive blood cultures.

Neutrophil CD64 expression was measured using l of whole blood incubated for minutes at room temperature with a saturating amount of fluorescein isothiocyanate (FITC)-conjugated anti-CD64 monoclonal antibody or isotype control (Leuko64 kit, Trillium Diagnostics, Scarborough, ME), followed by ammonium chloride-based red cell lysis. Samples were washed once and re-suspended in ml of phosphate-buffered saline with 0.1% bovine serum albumin. Flow cytometric analysis was accomplished using a Becton-Dickinson FACScan (Mountainview, CA) to collect log FITC fluorescence, log right-angle side scatter and forward scatter on a minimum of 50,000 leukocytes. Interassay standardization and neutrophil CD64 quantification were performed using FITC calibration beads (Leuko64 kit). Data analysis was performed using light scatter gating to define the neutrophil population, and the neutrophil CD64 Index was quantified as mean equivalent soluble fluorescence units using QuickCal for Winlist (Verity Software House, Topsham, ME) with a correction for nonspecific antibody binding by subtracting values for the isotype control [14]. This was expressed as an absolute value. Investigators checking and confirming the neutrophil CD64 results were blinded to the clinical data, including the blood culture results. Clinicians did not have access to the neutrophil CD64 values and these were not used to decide initiation or duration of antibiotic therapy.

Evaluation Studied

Evaluations were obtained by accessing the electronic medical record from January 2008 through June 2009. Each evaluation typically included a CBC, two peripheral blood cultures, and other optional cultures. A patient could undergo multiple sepsis evaluations during admission. Since a single evaluation represented a separate episode of suspected sepsis and could be treated independently, we therefore treated all evaluations equivalently in this manuscript. Evaluations were excluded if the CBC, neutrophil CD64, or blood culture tests were not provided in the patient record. A total of sepsis evaluations with complete hematologic, neutrophil CD64, and blood culture data were used for the analyses. (One evaluation, which was positive for was excluded due to the high neutrophil CD64 value that skewed the results.) Information about each sepsis evaluation included (1) sepsis diagnosis type, (2) day of life that the evaluation was performed, and (3) CBC data and neutrophil CD64 expression. Ten biomarkers were included in the analysis: Age, WBC, Hgb, Hct, Plt, Segs, Bands, Lymph, Mono, and CD64. Additional details about the laboratory and clinical data were recently published [27], [30].

Defining Sepsis Outcome

Individual sepsis evaluations with positive blood cultures were diagnosed as culture-proven sepsis according to the current National Healthcare Safety Network definitions for laboratory-confirmed bloodstream infections [31]. Individual sepsis evaluations with positive sepsis scores were categorized as clinical sepsis [8], [14]. This might include infants with other infectious diagnoses that were not accompanied by a positive blood culture, such as pneumonia, urinary tract infection, and necrotizing enterocolitis.

Three groups of evaluations were defined, and each evaluation was assigned to only one of the three groups. Group consisted of evaluations with a positive blood culture. Group with “suspected sepsis” consisted of evaluations, where the patients lacked a definitive positive blood culture, but the clinical diagnosis was unable to rule out bacterial infection. Group consisted of evaluations, for which either the blood culture or clinical diagnosis showed no evidence of infection.

Data Preprocessing

First, each evaluation , with data , is categorized as septic (groups and above) and nonseptic (group ), where is a real-valued vector with components (biomarkers) and is the total number of evaluations. For convenience, we labeled each evaluation using the variable , where is the label for the septic group and is given to each in the nonseptic group. To standardize the range of independent biomarkers, we normalized the real-valued data , a matrix, to have zero mean and unit standard deviation for each biomarker [32]: (2)where and are matrices, and are the mean value and standard deviation of for each biomarker.

Canonical Correlation Analysis

CCA is a multivariate statistical tool that facilitates the study of interrelationships among multiple variables [19], [33]. A linear combination of variables can be chosen by CCA such that the correlation between two sets of data is maximized [34]. In our studies, the two sets of data are the sepsis score and each distinct -combination of the biomarkers in the data matrix . In this investigation, CCA is used to identify the set of -biomarkers most correlated as a group to the sepsis score. Please see Supplementary Text S1 for more details.

To explore the redundancy among the biomarkers, we calculated the correlations between each possible -combination of and using CCA. By varying from (single biomarker) to (all biomarkers), we selected the specific set of biomarkers that possessed the highest correlation with for each . The sets of biomarkers that had the largest correlation with the sepsis score and their corresponding correlation coefficients are shown in Table 1.

Sparse Support Vector Machines (SSVM)

We applied the SSVM ensemble method to build a classifier for each of the CCA-selected -combination of biomarkers selected by CCA. A linear support vector machine (SVM) is a widely used classifier, which finds the hyperplane that separates high-dimensional data with maximum margin by categories. The search of this hyperplane can be translated into the following optimization problem: (3)where is the -norm of a vector, which induces the sparsity in the weight vector [17]. We refer to the solution of Equation (3) as a sparse support vector machine (SSVM) following Ref. [18]. Note that splitting the classes in the objective function allows for unbalanced sample sizes.

Due to the limited size and noise of our data, a bootstrap aggregation method was applied to build an ensemble of SSVM classifiers using the following procedure [35], [36]:

1. The data set is randomly divided into a learning set and a test set . is one third of the data.
2. Based on the bootstrap aggregation method, a bootstrap training set is randomly selected from the original learning set with replacement. That is, has the same number of samples as the original training set , but with several training samples appearing multiple times. Each bootstrap set contains unique samples of the original training set . By repeating this process times, an ensemble of classifiers , with , is built by the SSVM. To have the same total cost for both false positives and false negatives, the parameters and of the SSVM are chosen according to with since the results are not sensitive to the overall scale of .
3. The final classification is obtained by calculating the mean of the ensemble of classifiers.
4. The random division of the data into and is repeated times, after which we calculate the mean and standard deviation. We used the same random divisions of the training and test sets for each .

Calculation of Statistical Measures

The statistical measures of the performance of a classifier are measured using ACC, TPR, TNR, NPV, and PPV. For the sake of completeness, we include their definitions:(4a)(4b)(4c)(4d)(4e)

Here, these statistical measures are calculated for each one of the random divisions of test sets by the classifier built on the bootstrap aggregation method. Their mean and standard deviation are calculated from the groups obtained from the random divisions.

Clinical Issues in Sepsis Diagnosis

The problems associated with confirming sepsis with positive blood cultures, as mentioned earlier, has led clinicians to investigate alternate approaches for confirmation of diagnosis of blood-cluster negative or clinical sepsis and prevention of missed or under treatment of neonates with antibiotics. Clinical parameters are notorious for their non-specific nature in detecting infection, especially in premature neonates; however, a scoring system based on a 7-item weighted clinical score has been suggested [37]. In real world settings, most clinicians rely on clinical judgment, in concert with specific hematological criteria, to identify infants with sepsis. The hematological criteria have usually included ANC, ABC, IT-ratio, and platelet counts, as was done in the present study [6], [14], [38]–[43]. Unfortunately, this approach has not proven very reliable due to the inherent subjective nature of the clinical assessment and the variability of the hematological parameters secondary to physiological derangements and non-infectious medical conditions, and has led to over-treatment of neonates.

It has been suggested that the addition of specific molecular markers might improve diagnostic accuracy of neonatal sepsis. Among the acute-phase reactants, CRP is probably the most well studied, but its value for diagnostic accuracy has had mixed results [7], [44]. Among the newer ones, procalcitonin [45]–[49] and neutrophil CD64 [14], [15] have shown promise. Neutrophil CD64 values have been reported to be sustained for at least 24 hours in neonates with sepsis [50].

Several studies have investigated the usefulness of the CD64 Index in the NICU population, albeit in much smaller cohorts, but with promising results in both the preterm and term populations, as well as in cases of both early-onset sepsis and late-onset sepsis [14], [51]–[56]. Recently, studies have suggested that the diagnostic accuracy of neutrophil CD64 is superior to the IT-ratio [57] and CRP [58] for the early detection of neonatal sepsis. Secondly, testing can be done on the same sample sent for a CBC evaluation as it requires only 50 l of blood. Thirdly, the CD64 results can be made available within hours of the CBC, since most clinical laboratories in the developed and some developing countries have flow cytometry technology. Furthermore, standard cell counters which use flow cytometry have the potential to incorporate anti-CD64 antibodies and software to provide an even more rapid enumeration of CD64 indices nearly simultaneous with CBC results. Hence, we believe that a scoring system that can incorporate the common CBC parameters with the neutrophil CD64, as was done with our analyses, would provide objective criteria for recognizing neonatal sepsis and guidance for initiation and/or early termination of antibiotic therapy. Additional independent validation of our results is needed before incorporation of our diagnostic sepsis score can be recommended for routine clinical use.

Identification of the Optimal Subset of Biomarkers

We have proposed a new approach for biomarker identification based on the integrated use of CCA and SSVM on a labeled data set. We found that for the neonatal sepsis data our approach produced the optimal set of hematological biomarkers for all possible . We validated our results by conducting an exhaustive search of all combinations of biomarkers and ranking them based on their classification accuracy. These results showed that our approach produced either the absolute top combination, or a combination of biomarkers with statistically indistinguishable performance. Although this study explored a relatively small set of biomarkers, the CCA approach can be applied to potentially much larger sets by exploiting the relative weighting of the features (see Equations S1 and S2) and selecting only the most important features. From Equation S3, CCA requires finding the pseudo-inverse of a matrix, where is the number of biomarkers. Even without invoking sparse methods, one can easily investigate systems with of order .

Our approach identifies Bands as the most significant biomarker in our set for detecting neonatal sepsis. The next four most sigificant hematological biomarkers, in order of importance, appear to be Plt, neutrophil CD64, WBC, and Segs. We note that, as illustrated in Figure S1, the reason for the significance of Bands may be attributed to the fact that it is highly correlated with subjects with a negative diagnosis while much less correlated with subjects with a positive diagnosis. This could be related to the significant variation in Bands for sick individuals as evidenced by Table S1.

We explore LLR in addition to SSVM to corroborate our results. In each case we see that an exhaustive combinatorial evaluation of the classifiers determines that the CCA selected biomarkers were indeed optimal. See Figures S2 and S3 for a graphical summary of these numerical experiments. Additionally we found that the Forward Selection results were very similar to the biomarkers identified by CCA on this data set, i.e., Bands, Plt, Hgb, CD64 and Segs. CCA selected WBC and not Hgb for the best 5-combination. The classifiers using these two sets of biomarkers perform very similarly with a very slight edge to the CCA biomarkers. However, in general, forward selection is a greedy algorithm and it is possible that the sets of biomarkers identified by CCA and FS could be quite different. Classification algorithms such as SSVM or LLR can then assist in comparing and evaluating the selected biomarkers.

We propose that the results found in this investigation, in particular, the new sepsis scoring system, sets the stage for independent investigators to clinically validate these results using alternative sepsis databases. In particular, it will be interesting to ascertain whether this scoring system is also relevant for adults. It is also possible to envision modifying the scoring system based on new data related to alternative scenarios, e.g., septic adults infected by Gram negative microorganisms. Although we propose CCA in conjunction with SSVM as an approach for biomarker identification, the strength of the methodology lies in the exploitation of multivariate relationships within the data and other methods that do this also merit further exploration.