Figures
Abstract
Background
Accurate estimations of life expectancy are important in the management of patients with metastatic cancer affecting the extremities, and help set patient, family, and physician expectations. Clinically, the decision whether to operate on patients with skeletal metastases, as well as the choice of surgical procedure, are predicated on an individual patient's estimated survival. Currently, there are no reliable methods for estimating survival in this patient population. Bayesian classification, which includes Bayesian belief network (BBN) modeling, is a statistical method that explores conditional, probabilistic relationships between variables to estimate the likelihood of an outcome using observed data. Thus, BBN models are being used with increasing frequency in a variety of diagnoses to codify complex clinical data into prognostic models. The purpose of this study was to determine the feasibility of developing Bayesian classifiers to estimate survival in patients undergoing surgery for metastases of the axial and appendicular skeleton.
Methods
We searched an institution-owned patient management database for all patients who underwent surgery for skeletal metastases between 1999 and 2003. We then developed and trained a machine-learned BBN model to estimate survival in months using candidate features based on historical data. Ten-fold cross-validation and receiver operating characteristic (ROC) curve analysis were performed to evaluate the BNN model's accuracy and robustness.
Results
A total of 189 consecutive patients were included. First-degree predictors of survival differed between the 3-month and 12-month models. Following cross validation, the area under the ROC curve was 0.85 (95% CI: 0.80–0.93) for 3-month probability of survival and 0.83 (95% CI: 0.77–0.90) for 12-month probability of survival.
Conclusions
A robust, accurate, probabilistic naïve BBN model was successfully developed using observed clinical data to estimate individualized survival in patients with operable skeletal metastases. This method warrants further development and must be externally validated in other patient populations.
Citation: Forsberg JA, Eberhardt J, Boland PJ, Wedin R, Healey JH (2011) Estimating Survival in Patients with Operable Skeletal Metastases: An Application of a Bayesian Belief Network. PLoS ONE 6(5): e19956. https://doi.org/10.1371/journal.pone.0019956
Editor: Wafik S. El-Deiry, University of Pennsylvania, United States of America
Received: February 7, 2011; Accepted: April 8, 2011; Published: May 13, 2011
Copyright: © 2011 Forsberg et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors have no support or funding to report.
Competing interests: John Eberhardt (JE) is an employee and shareholder of DecisionQ Corporation, which is the manufacturer of software products used in this research. Neither JE, nor DecisionQ Corporation received compensation in the conduct of this study or the preparation of this paper. This does not alter the authors' adherence to all the PLoS ONE policies on sharing data and materials.
Introduction
“Doc, how long have I got?” For the physician, such questions related to life expectancy are among the most difficult to answer. An accurate estimation of survival is important, however, and can help set patient, family, and physician expectations. Clinically, the decision to operate on patients with skeletal metastases of the extremities, as well as the choice of surgical procedure, is predicated on a patient's estimated survival [1]. The goal of surgery in this setting is not to cure the disease, but to relieve pain and optimize functional mobility for the maximum amount of time. Thus, a considerable amount of effort has been spent identifying useful prognosticators for use in the metastatic setting.
Several independent predictors of survival have been identified for patients with metastatic bone disease of the extremities [2]–[8]. These include the primary oncologic diagnosis, Eastern Cooperative Oncology Group (ECOG) performance status score [9]; number of bone metastases, presence of visceral metastases, serum hemoglobin level, senior surgeon's estimate of survival [6]; appendicular, as opposed to axial, bone metastases [10]; and type of reconstructive procedure performed [5]. Although many prognostic factors are known, the predictability of survival in patients with metastatic bone disease remains alarmingly low—only 5–15% in the best-reported series [6]. Relationships between prognostic variables are difficult to interpret, and better means of prognostication are needed since there are currently no reliable methods for estimating survival in this patient population.
Bayesian classification, which includes Bayesian belief network (BBN) modeling, is a statistical method that represents conditional, probabilistic relationships between variables, or “features.” Once described, these relationships enable the development of a graphical n-dimensional structure, or model, which codifies all features, including the outcome(s), into a single hierarchical network. Bayesian classification also accounts effectively for data multi-dimensionality and uncertainty—a quality that enables BBN models to maintain their robustness in the context of incomplete or discordant clinical data. As such, BBNs have been successfully used to both model complex relationships, as well as to classify outcomes, in a variety of oncologic diagnoses [11]–[20]. The purpose of this study was to determine the feasibility of developing and training Bayesian classifiers designed to estimate survival in patients undergoing surgery for skeletal metastases involving the axial and appendicular skeleton.
Methods
After obtaining approval from the institutional review board of Memorial Sloan-Kettering Cancer Center, who issued a waiver of informed consent, we searched the institution-owned patient management database (Disease Management System, v.5.2, 1996; MSKCC) for all patients who underwent surgery for metastatic bone disease between 1999 and 2003. For each patient, we reviewed his or her medical records and imaging studies. These data, along with several other features, were used to construct the BBN model.
Fifteen candidate features were chosen based on their current clinical or historical association with survival in patients with operatively treated skeletal metastases. These included the following: age at the time of surgery, race, sex, estimated glomerular filtration rate (mL/min/1.73 m2), serum calcium concentration (mg/dL), serum albumin concentration (g/dL), indication for surgery (impending or completed pathologic fracture), number of bone metastases (solitary or multiple), surgeon's estimate of survival (postoperatively, in months), presence or absence of visceral metastases, presence or absence of lymph node metastases, prior chemotherapy (yes or no), preoperative hemoglobin (mg/dL, on admission, prior to transfusion, if applicable), absolute lymphocyte count (K/µL), and the primary oncologic diagnosis. The oncologic diagnosis was classified into three groups according to the method described by Katagiri et al. [21], but with some modifications. Briefly, breast, prostate, renal cell, and thyroid carcinoma, multiple myeloma, and malignant lymphoma were included in Group 3; sarcomas and other carcinomas were included in Group 2; and lung, gastric, and hepatocellular carcinoma and melanoma were included in Group 1.
The following definitions were used in this study. An impending pathologic fracture was one in which the degree of bone and/or cortical disruption warranted, in the opinion of the treating surgeon (PJB or JHH), prophylactic surgical stabilization to prevent fracture. A completed pathologic fracture was one in which a cortical lesion had caused a change in bone length, alignment, rotation, or loss of height as determined by imaging. The surgeon's estimate of survival was made preoperatively by the treating surgeon (PJB or JHH) after reviewing the patients' medical records and imaging studies, obtained a complete medical history, and performed a thorough physical examination. Biopsy-proven or clinically obvious metastases to organs within the chest or abdomen were considered visceral metastases. Only biopsy-proven metastases to the lymph nodes were considered indicative of lymph node involvement. Finally, a patient was considered as having undergone prior chemotherapy, if he or she had received any chemotherapy at any time for the current, active oncologic diagnosis.
Conventional Statistical Analysis
Conventional statistical analysis was performed using SAS software (v.9.2, SAS Institute Inc., Cary, NC, USA). Since we modified a previously described diagnosis grouping method, overall survival of patients in each diagnosis group was compared using the Kaplan-Meier method with log-rank assessment.
Bayesian Statistical Analysis and Model Development
The BBN models were developed using commercially available machine-learning algorithms (FasterAnalytics, DecisionQ, Washington, DC, USA) that automatically learn network structures and joint probabilities from the training data.
The training data set included all cases identified from the patient management database during the study period. All 15 candidate features were considered. Features containing continuous variables were converted into categorical variables by equal-area binning based on prior distributions. The machine-learning algorithm uses a scoring formula that balances goodness-of-fit against robustness using a parsimony metric to reduce the risk of overfitting the final model to the training data set. To refine the model, we used a step-wise training process. Quantitative and qualitative assessments were used to optimize variable preparation and selection. Unrelated and confounding features were then pruned from the preliminary models to produce the final models.
We trained the BBN to specify network structure and prior probability distributions to develop classifiers of estimated survival at 3 and 12 months, which we consider to be useful discriminators for surgical decision-making. We trained a separate classifier for each outcome measure (survival >3 months and survival >12 months) because, as a parametric modeling methodology, BBNs are not well suited to provide discrete estimates in time. Also, training two separate models eliminates the possibility of outcome measures acting as confounding features within the same BBN.
Because BBN models are directed graphs of conditional dependence, the network structure can be portrayed graphically to illustrate the conditional interdependence of the features. First-degree predictors are defined as those nodes that share edges with the outcome of interest, while second-degree predictors are those nodes that share edges with the first-degree predictors. Inference tables were calculated for both models depicting posterior estimates of probability for each possible permutation and the expected outcome, survival greater than 3 months and 12 months, respectively.
Internal Validation
Ten-fold cross-validation was performed to assess the robustness of the final 3-month and 12-month models. Data were randomized and divided into 10 matching train-and-test sets. Each train-and-test set consisted of a training set composed of 90% of patient records and a test set composed of the remaining 10% of records. Each matching set was unique, and there was no overlap between the independent test sets. A BBN model was trained, using each training set, by applying the same parameters as the final models, then tested on the corresponding test set. A receiver operating characteristic (ROC) curve was plotted for each test to evaluate the classifier's accuracy and the model's robustness. The ROC curve is a graphical plot of sensitivity vs. 1-specificity at all discrimination threshold levels. The area under the ROC curve (AUC) was then calculated for each BBN model to assess its overall accuracy and robustness.
Results
A total of 189 consecutive patients were identified and, thus, used for this analysis. Median follow-up was 8 months (interquartile range [IQR] 2, 22), which was adequate for determining overall survival up to and including either the 3-month or 12-month postoperative time points. Median patient age was 62 years (54, 72). Most patients were women (55.1%), and most were white, non-Hispanic (85.2%). Most patients also had visceral metastases (60.3%), multiple bone metastases (71.0%), and prior systemic therapy (56.1%). A few patients had lymph node involvement (18.8%). When classified according to diagnosis group, most patients were in Group 3 (54.5%), followed by Group 1 (27.2%), and then Group 2 (18.2%). Regarding overall survival (Fig. 1), 58 patients (30.7%) survived less than 3 months, 53 (28.0%) survived 3–12 months, and 78 (41.3%) survived more than 12 months. Baseline (posterior) distributions represented as proportions are depicted in Fig. 2.
The overall survival of patients in Group 1 was significantly lower than that of patients in Groups 2 and 3 at the 3-month time point∞ (p<0.0001, log-rank test). Overall survival was significantly different between all groups at the 12-month time point* (p<0.0001, log-rank test).
As shown, there are five first-degree predictors of 3-month survival: the surgeon's estimate of survival (“surgeon_estimate_of_survival”), preoperative hemoglobin concentration (“hemoglobin”), preoperative absolute lymphocyte count (“absolute_lymphocyte_count”), ECOG performance status (“ECOG”), and the presence of a completed pathologic fracture (“completed_path_fx”). The network structure indicates that the primary oncologic diagnosis (“dx_grouping”) and the presence of visceral metastases (“visceral_mets”) are both first-degree associates of the surgeon's estimate node.
The features comprising the final models are described in Table 1. First-degree predictors differed between the two models. In the 3-month model (Fig. 2), the surgeon's estimate of survival, hemoglobin concentration, absolute lymphocyte count, completed pathologic fracture, and ECOG performance status were first-degree predictors of survival. In the 12-month model (Fig. 3), only the surgeon's estimate of survival, hemoglobin concentration, number of bone metastases, and the diagnosis group were first-degree predictors of survival. In the 3-month model, the diagnosis group and the presence of visceral metastases were first-degree associates of the surgeon's estimate node. In contrast, in the 12-month model, ECOG performance status and presence of visceral metastases were first-degree associates of the surgeon's estimate node.
As shown, there are four first-degree predictors of 12-month survival: the surgeon's estimate of survival (“surgeon_estimate_of_survival”), preoperative hemoglobin concentration (“hemoglobin”), the number of bone metastases (“bone_mets”), and the primary oncologic diagnosis (“dx_grouping”). In contrast to the 3-month model, the network structure of the 12-month model indicates that the ECOG performance status (“ECOG”) and the presence of visceral metastases (visceral_mets) are both first-degree associates of the surgeon's estimate node.
Cross-validation ROC curve analysis showed that both the 3-month and 12-month models were robust. Mean AUC for 3-month probability of survival was 0.85 (95% CI: 0.80–0.93), and mean AUC for 12-month probability of survival was 0.83 (95% CI: 0.77–0.90). Inference tables were calculated for 3-month survival (Table 2) and 12-month survival (Table 3). Posterior estimates of probability of 3-month survival ranged from 0.8–96.7%, while posterior estimates of probability of 12-month survival ranged from 0.9–99.2%. In each case, we used the estimated case frequency to select the ten most likely inferential cases, since there were 256 and 128 potential permutations for the 3- and 12-month models respectively.
Discussion
Because predictive models permitting individualized estimation of survival are lacking, we trained two full machine-learned BBN models, using observed clinical data, to estimate survival in patients with operable skeletal metastases. These models were shown to be robust on ten-fold cross-validation. We also characterized the importance of the surgeon's estimate of survival, and we believe that it should be included in future iterations of this model, whenever possible. However, if the surgeon's estimate is not included, ECOG performance status, the oncologic diagnosis group, and the presence of visceral metastases may be used to estimate this important subjective feature.
In the present analysis, we used the 3-month and 12-month time points, since we consider them to be useful discriminators for surgical decision-making. We believe that survival less than 3 months is a relative contraindication to surgical management of certain impending pathologic fractures, particularly those in the upper extremity. If surgical stabilization is deemed to be necessary, shorter life expectancies (3–12 months) are thought to warrant less-invasive stabilization procedures that do not require prolonged rehabilitation periods. On the other hand, longer life expectancies (≥12 months) warrant more-durable reconstruction procedures, which are associated with significant operative morbidity and longer rehabilitation times. Thus, less-invasive stabilization techniques may be appropriate for patients with an estimated survival of less than 12 months, while more-durable reconstruction options may be considered for those with an estimated survival longer than 12 months. These concepts and the surgical techniques are described elsewhere [22].
Unfortunately, although modeling and, thus, estimating survival in patients with metastatic disease is important in the surgical management of these patients, it has proven to be challenging [2], [5], [6], [8]. For instance, there is no consensus regarding which variables should be included in survival estimation models. In an attempt to codify several known independent predictors of survival, Nathan et al. developed a conventional, regression-derived nomogram based on 191 patients with operable skeletal metastases [6]. Variables, collected prospectively, included ECOG performance status, number of bone metastases, presence of visceral metastases, and serum hemoglobin concentration. Evaluation in a five-patient test set yielded an accurate survival estimate in two patients, but this method has not yet been internally or externally validated.
A closer look at Nathan et al.'s results gives us some insight into the challenges inherent to predicting survival in these patients [6]. Linear regression analysis revealed that, in terms of accuracy, the “attending surgeon's [subjective] prediction” of survival was superior to any other covariate included in their analysis. The authors attributed this finding to the attending surgeon's extensive experience in the field, but no further conclusions were drawn. Other studies, including a systematic review of physicians' survival predictions, also emphasize subjective assessments and cite the importance of the physician's role in estimating patient survival [23]–[25].
In the context of the current study, what makes the surgeon's estimate of survival so important? Clearly, an estimate of survival is made after reviewing the patient's medical records and imaging studies and performing a thorough physical examination on the patient. Some variables considered by the surgeon are objective and thus quantifiable, such as ECOG performance scores, laboratory results, and radiographic findings. However, others are subjective and thus not quantifiable, such as the patient's appearance of sickness, the patient's demeanor, and the surgeon's “gut feeling” following the consultation. These subjective observations are unlikely to be useful as individual covariates in the traditional sense, but we believe that they are contained within the surgeon's estimate, which was found to be an important first-degree predictor in both survival models (Figs. 2 and 3). Historically, the physician's assessment appears to be important when estimating survival [6], [23]–[25], and if the subjective observations listed above are indeed represented by the surgeon's estimate, the results of the present study support this claim.
Fortunately, the BBN method may help us to better understand the complex relationships that exist between objective and subjective features. The hierarchy of conditional dependence, identified by the BBN, defines how individual features known to be associated with survival relate to one another. When graphically displayed, the network structure depicts these relationships. For example, we found that the surgeon's estimate is a first-degree predictor of overall survival at both 3 months and 12 months. However, the nodes associated with the surgeon's estimate differed between the models. In the 3-month model, the oncologic diagnosis group and the presence of visceral metastases were first-degree associates of the surgeon's estimate node. In contrast, in the 12-month model, ECOG performance status and the presence of visceral metastases were first-degree associates. This suggests that in short-term survivors, represented by the 3-month model, specific oncologic diagnosis and the presence of visceral metastases are most influential in the development of a survival estimate by the senior surgeon, while in longer-term survivors, represented by the 12-month model, ECOG performance status and the presence of visceral metastases are most important in the development of the surgeon's estimate of survival. Other features, such as the number of bone metastases, hemoglobin concentration, absolute lymphocyte count, and completed pathologic fracture, may also inform the surgeon's estimate, but they have significant prognostic value of their own, as shown in both model structures (Figs. 2 and 3). This is further supported by the observation that a BBN model that included the surgeon's estimate and these independent factors (the number of bone metastases, hemoglobin concentration, absolute lymphocyte count, and completed pathologic fracture) outperformed a BBN model containing the surgeon's estimate alone (data not shown). Nevertheless, in order to capture the subjective, yet important, elements of the clinical evaluation, we believe that future predictive models designed to estimate survival in patients with operatively treated skeletal metastases should include a surgeon's estimate of survival.
This cohort of surgically treated patients with metastatic disease involving the axial and appendicular skeleton is among the largest reported in the literature [1], [5], [6], [10], [21]. As such, it is well suited for prognostic model development. Nevertheless, the limitations of this study are those inherent to a feasibility study. No comparisons were made to existing models or nomograms, and no external validation was performed. Validation of these models in an independent data set followed by an appropriate comparison to existing models is the next logical step in the evaluation of this method. The ultimate goal being to provide an accurate, personalized answer to the difficult question, “Doc, how long have I got?”
Acknowledgments
We thank Dr. Mithat Gönen (Department of Epidemiology and Biostatistics, Memorial Sloan-Kettering Cancer Center, New York, NY) for his insight into both conventional and Bayesian statistics, as well as Isobel Lewis and Neha Kaul, whose dedication and organizational skills made this project possible. We also thank Lionel Santibañez for his superb editorial assistance.
Author Contributions
Conceived and designed the experiments: JAF JE JHH. Performed the experiments: PJB JHH. Analyzed the data: JAF JE RW. Contributed reagents/materials/analysis tools: JAF JE PJB JHH. Wrote the paper: JAF JE PJB RW JHH.
References
- 1. Wedin R (2001) Surgical treatment for pathologic fracture. Acta Orthop Scand Suppl 72: 2p.1–29
- 2. Hansen BH, Keller J, Laitinen M, Berg P, Skjeldal S, et al. (2004) The Scandinavian Sarcoma Group Skeletal Metastasis Register. Survival after surgery for bone metastases in the pelvis and extremities. Acta Orthop Scand Suppl 75: 11–15.
- 3. Hartsell WF, Desilvio M, Bruner DW, Scarantino C, Ivker R, et al. (2008) Can physicians accurately predict survival time in patients with metastatic cancer? Analysis of RTOG 97-14. J Palliat Med 11: 723–728.
- 4. Lin PP, Mirza AN, Lewis VO, Cannon CP, Tu SM, et al. (2007) Patient survival after surgery for osseous metastases from renal cell carcinoma. J Bone Joint Surg Am 89: 1794–1801.
- 5. Narazaki DK, de Alverga Neto CC, Baptista AM, Caiero MT, de Camargo OP (2006) Prognostic factors in pathologic fractures secondary to metastatic tumors. Clinics 61: 313–320.
- 6. Nathan SS, Healey JH, Mellano D, Hoang B, Lewis I, et al. (2005) Survival in patients operated on for pathologic fracture: implications for end-of-life orthopedic care. J Clin Oncol 23: 6072–6082.
- 7. Saad F, Lipton A, Cook R, Chen YM, Smith M, et al. (2007) Pathologic fractures correlate with reduced survival in patients with malignant bone disease. Cancer 110: 1860–1867.
- 8. Schneiderbauer MM, von Knoch M, Schleck CD, Harmsen WS, Sim FH, et al. (2004) Patient survival after hip arthroplasty for metastatic disease of the hip. J Bone Joint Surg Am 86-A: 1684–1689.
- 9. Oken MM, Creech RH, Tormey DC, Horton J, Davis TE, et al. (1982) Toxicity and response criteria of the Eastern Cooperative Oncology Group. Am J Clin Oncol.
- 10. Sugiura H, Yamada K, Sugiura T, Hida T, Mitsudomi T (2008) Predictors of survival in patients with bone metastasis of lung cancer. Clin Orthop Relat Res 466: 729–736.
- 11. Adamina M, Tomlinson G, Guller U (2009) Bayesian statistics in oncology: a guide for the clinical investigator. Cancer 115: 5371–5381.
- 12. Cheng SH, Horng CF, Clarke JL, Tsou MH, Tsai SY, et al. (2006) Prognostic index score and clinical prediction model of local regional recurrence after mastectomy in breast cancer patients. Int J Radiat Oncol Biol Phys 64: 1401–1409.
- 13. Damato B, Eleuteri A, Fisher AC, Coupland SE, Taktak AF (2008) Artificial Neural Networks Estimating Survival Probability after Treatment of Choroidal Melanoma. Ophthalmology.
- 14. Heriot AG, Tekkis PP, Smith JJ, Cohen CR, Montgomery A, et al. (2006) Prediction of postoperative mortality in elderly patients with colorectal cancer. Dis Colon Rectum 49: 816–824.
- 15. Jerez-Aragones JM, Gomez-Ruiz JA, Ramos-Jimenez G, Munoz-Perez J, Alba-Conejo E (2003) A combined neural network and decision trees model for prognosis of breast cancer relapse. Artif Intell Med 27: 45–63.
- 16. Levitt SH, Aeppli DM, Nierengarten ME (1995) The importance of local control in the conservative treatment of breast cancer. Acta Oncol 34: 839–844.
- 17. Lisboa PJ, Taktak AF (2006) The use of artificial neural networks in decision support in cancer: a systematic review. Neural Netw 19: 408–415.
- 18. Orr RK (1999) The impact of prophylactic axillary node dissection on breast cancer survival–a Bayesian meta-analysis. Ann Surg Oncol 6: 109–116.
- 19. Taktak AF, Fisher AC, Damato BE (2004) Modelling survival after treatment of intraocular melanoma using artificial neural networks and Bayes theorem. Phys Med Biol 49: 87–98.
- 20. Stojadinovic A, Peoples GE, Libutti SK, Henry LR, Eberhardt J, et al. (2009) Development of a clinical decision model for thyroid nodules. BMC Surg 9: 12.
- 21. Katagiri H, Takahashi M, Wakai K, Sugiura H, Kataoka T, et al. (2005) Prognostic factors and a scoring system for patients with skeletal metastasis. J Bone Joint Surg Br 87: 698–703.
- 22.
DeVita VT, Lawrence TS, Rosenberg SA, Weinberg RA, DePinho RA (2008) DeVita, Hellman, and Rosenberg's Cancer: principles & practice of oncology. Lippincott Williams & Wilkins.
- 23. Glare P, Virik K, Jones M, Hudson M, Eychmuller S, et al. (2003) A systematic review of physicians' survival predictions in terminally ill cancer patients. BMJ 327: 195–198.
- 24. Mackillop WJ, Quirt CF (1997) Measuring the accuracy of prognostic judgments in oncology. J Clin Epidemiol 50: 21–29.
- 25. Muers MF, Shevlin P, Brown J (1996) Prognosis in lung cancer: physicians' opinions compared with outcome and a predictive model. Thorax 51: 894–902.