Guidelines for the prevention of coronary heart disease (CHD) recommend use of Framingham-based risk scores that were developed in white middle-aged populations. It remains unclear whether and how CHD risk prediction might be improved among older adults. We aimed to compare the prognostic performance of the Framingham risk score (FRS), directly and after recalibration, with refit functions derived from the present cohort, as well as to assess the utility of adding other routinely available risk parameters to FRS.
Among 2193 black and white older adults (mean age, 73.5 years) without pre-existing cardiovascular disease from the Health ABC cohort, we examined adjudicated CHD events, defined as incident myocardial infarction, CHD death, and hospitalization for angina or coronary revascularization.
During 8-year follow-up, 351 participants experienced CHD events. The FRS poorly discriminated between persons who experienced CHD events vs. not (C-index: 0.577 in women; 0.583 in men) and underestimated absolute risk prediction by 51% in women and 8% in men. Recalibration of the FRS improved absolute risk prediction, particulary for women. For both genders, refitting these functions substantially improved absolute risk prediction, with similar discrimination to the FRS. Results did not differ between whites and blacks. The addition of lifestyle variables, waist circumference and creatinine did not improve risk prediction beyond risk factors of the FRS.
The FRS underestimates CHD risk in older adults, particularly in women, although traditional risk factors remain the best predictors of CHD. Re-estimated risk functions using these factors improve accurate estimation of absolute risk.
Citation: Rodondi N, Locatelli I, Aujesky D, Butler J, Vittinghoff E, et al. (2012) Framingham Risk Score and Alternatives for Prediction of Coronary Heart Disease in Older Adults. PLoS ONE 7(3): e34287. doi:10.1371/journal.pone.0034287
Editor: Weili Zhang, FuWai hospital, Chinese Academy of Medical Sciences, China
Received: January 8, 2012; Accepted: February 26, 2012; Published: March 28, 2012
Copyright: © 2012 Rodondi et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Guidelines for the prevention of coronary heart disease (CHD) recommend the use of risk scores to identify adults at higher risk of CHD for whom preventive therapy–e.g., by lipid lowering drugs–has higher absolute benefits . Several scoring systems exist to help clinicians assess the 10-year CHD risk , , , with the Framingham risk score (FRS)  the most widely used. US Guidelines for the prescription of lipid-lowering drug therapy  and aspirin in primary prevention  are based on the risk estimations provided by the FRS.
Most risk scores were developed in white middle-aged populations , , . Thus, it is uncertain whether risk estimates based on these scores can be generalized to the elderly. The FRS, for example, was developed in a white middle-aged population with a mean age of 49 years and included persons as young as 30 and none older than 74 . Actual risk prediction with FRS might perform less well in older adults compared to middle-aged adults, and some traditional risk factors have weaker associations with CHD risk in the elderly; for example, total and LDL-cholesterol are strong cardiovascular risk factors in middle-aged but not in older adults .
As it remains unclear whether and how CHD risk prediction might be improved in the growing population of elderly  to facilitate primary prevention strategies, we aimed to compare the prognostic performance of 1) the FRS, directly and 2) after recalibration , and 3) with functions derived from the Health ABC Study, a cohort of elderly white and black men and women . We also aimed to assess 4) the utility of adding routinely available lifestyle and simple laboratory variables not part of the FRS but which have been shown to predict CHD in older adults, such as creatinine , glucose  and lifestyle factors (alcohol consumption , physical activity ).
Participants were part of the Health, Aging, and Body Composition Study (Health ABC Study), a population-based cohort of 3075 community-dwelling men and women, aged 70–79 during the study enrollment period in 1997–1998. Participants were identified from a random sample of white and all black Medicare-eligible adults living in designated zip codes areas surrounding Pittsburgh, PA, and Memphis, TN. Eligibility criteria at baseline included the ability to walk ¼ mile, up 10 stairs without rest and perform basic activities of daily living independently . All participants gave written informed consent and the Pittsburgh and Memphis Institutional Review Boards approved the protocol.
Among the 3075 participants, we excluded 841 who had overt cardiovascular disease (CVD) at baseline, defined as diagnosis of CHD (angina, prior myocardial infarction, angioplasty of coronary arteries or coronary artery surgery), stroke or transient ischemic attack, peripheral arterial revascularization, carotid artery disease, heart failure or having a pacemaker. We also excluded 41 participants with missing data for any of the traditional cardiovascular risk factors. The final sample for our analyses was 2193 participants.
Cardiovascular risk factors.
Participants reported smoking history and were classified as never, current, or former smoker. Fasting total cholesterol, HDL-cholesterol, and blood pressure were measured as previously described . Hypertension was defined as self-report and use of anti-hypertensive medications, or measured blood pressure ≥140 and/or ≥90 mm Hg. Diabetes was defined as self-reported medical diagnosis and/or using any hypoglycemic medication . Physical activity was assessed by questionnaire about all types of walking and exercise performed in the prior week .
During 8-year follow-up, we assessed incident CHD events and mortality among participants without overt CVD at baseline . Using algorithms mirroring those of the Cardiovascular Health Study , diagnoses and cause of death were adjudicated until 2006–2007 based on interview, review of all hospital records, death certificates, and other documents by a panel of clinicians. CHD events included nonfatal myocardial infarction or coronary death (corresponding to “hard” events, as defined in the current FRS ), and hospitalization for angina or revascularization (coronary angioplasty or surgery) .
The FRS predicts 10-year CHD risk based on a Cox model estimated using data from the Framingham Heart Study . The Framingham cohort included 5345 subjects aged 30–74 years at the time of their examination in 1971–1974. For this analysis, we used the sex-specific Framingham equations of Wilson , because they include diabetes, a strong independent CHD risk factor , . This FRS Cox model includes age, total and HDL cholesterol, blood pressure, diabetes, and smoking status.
In this study, we compared the prognostic performance of the FRS, directly and after recalibration (taking into account different prevalence of risk factors and underlying rates of developing CHD), with functions entirely derived from the Health ABC cohort, similar to previous studies . Analyses were stratified by gender. We first estimated the FRS using regression coefficient estimates and values of the risk factor means reported by Wilson . To account for the shorter follow-up in the Health ABC study and to avoid extrapolation beyond the range of the data , we examined 7.5-year risk and adapted accordingly the estimated baseline survival function used in computing the FRS. Participants who died from non-CHD death were censored at the time of death.
We then examined whether the predictive performance of the FRS could be improved with recalibration or with refitting model coefficients. For the recalibrated version of the FRS , we re-estimated predicted risks for Health ABC by retaining the original coefficient estimates reported by Wilson  but adapted the risk factor means to the present cohort and the Kaplan Meier estimate of the baseline survival function of Health ABC data. For the refit version of the FRS (the “Health ABC function”), we estimated the regression coefficients with a Cox model fitted to the Health ABC data, obtaining an estimated predicted risk entirely based on Health ABC data. In this model, some adjacent risk factor categories were combined to avoid cells with limited numbers of events and/or unpredictive trends.
To compare prediction of these three risk models, we examined different statistical measures. To assess discrimination, we used Harrell's C-index , an adaptation of the C-statistic an adaptation of the C-statistic or area under the ROC curve for use with survival data. As the model validation for Health ABC functions was performed on the same dataset used for estimating the Cox model and the sample included too few events for split-sample validation, we calculated an optimism-corrected C-index using bootstrap resampling  with 1000 replications . To assess model calibration, we used Parzen's adaptation  of the Hosmer-Lemeshow test to the Cox model.
In exploratory analysis, we sought to determine whether alternative sets of predictors would improve risk prediction. To evaluate the utility of adding to the FRS different lifestyle and simple laboratory variables, we initially considered predictor variables with p<0.20 in unadjusted Cox models for CHD events in Health ABC data. We then used three model selection procedures: a backward selection with a retention criterion of p<0.10, and two forward stepwise selection procedures minimizing the Akaike Information Criterion (AIC) and the Bayesian Information Criterion (BIC), respectively . In these models, total cholesterol, HDL-cholesterol, and blood pressure were modeled as continuous predictors. Statistical analyses were performed using the software R, version 2.9.1 (R Foundation for Statistical Computing, Vienna, Austria).
At baseline, the mean age of the study participants was 73.5 years; 55% were women, and 41% were black (Table 1). The mean 10-year risk based on the FRS was 14.9%. Most participants had a 10-year CHD risk ranging from 5 to 19.9%.
Table 1. Baseline characteristics and unadjusted associations with incident CHD events (n = 2193; number of CHD events = 351).doi:10.1371/journal.pone.0034287.t001
During a median follow-up of 8.3 years (maximum, 10.2 years), 351 participants developed a CHD event (197 of which had a “hard” CHD event). In unadjusted analyses, all traditional cardiovascular risk factors were associated with CHD events except for total cholesterol and LDL-cholesterol (Table 1). Abdominal circumference, glucose, and creatinine were also associated with CHD events, but not glomerular filtration rate, alcohol use or physical activity levels. Results were similar for hard CHD events with larger confidence intervals because of lower number of events (data not shown), except that the association with abdominal circumference disappeared (HR = 1.00).
Number of participants in different risk factor categories and CHD events are shown in Table 2 for women and Table 3 for men. The original FRS had poor discrimination in these older adults (C-index: 0.577 in women; 0.583 in men). Using risk factors as continuous variables yielded similar C-indexes. Calibration of the original FRS was also poor in older adults (Figure 1), particularly among women, for whom the absolute risk was underestimated by 51% (vs. 8% in men, Table S1). Recalibration of the FRS improved calibration, particularly for women, and produced a better match between observed and expected CHD risk (Figure 1, Table S1). Statistically significant differences between observed and expected risks across deciles remained, at least in in women (p value remaining <0.05, larger p-values indicating better calibration), with an overestimation of the predicted risk for those above the median risk by a factor of 1.4 for women and 1.3 for men. For both genders, the Health ABC function significantly improved calibration (Figure 1). For discrimination, the C-index for the Health ABC function, after correction for optimism, was comparable to the C-index of the FRS (p = 0.54 for women and 0.90 for men, Tables 2 and 3). Total cholesterol and age2 were not predictive in women and were therefore omitted in the Health ABC function. Overall, results did not differ between whites and blacks. C-indexes for the unmodified FRS, the recalibrated FRS and the Health ABC function stratified by gender did not significantly differ between whites and blacks (all p for interaction >0.20; C-indexes ranging from 0.550 to 0.603). Calibration became reasonable (with p>0.20 for comparison of observed with expected) in white men for the unmodified FRS and in white men and women for the recalibrated FRS, but the best calibration remained for the Health ABC function (p>0.20 for comparison of observed with expected in the four subgroups stratified by race and gender).
Figure 1. Predicted risk of CHD events at 7.5 years according to original Framingham functions, recalibrated Framingham functions and Health ABC functions.doi:10.1371/journal.pone.0034287.g001
Table 2. Discrimination and calibration of Framingham functions (FRS), recalibrated FRS and Health ABC function in women (n = 1212).doi:10.1371/journal.pone.0034287.t002
Table 3. Discrimination and calibration of Framingham functions (FRS), recalibrated FRS and Health ABC function in men (n = 981).doi:10.1371/journal.pone.0034287.t003
We used a variety of model selection procedures when considering the addition of routinely available measures not included in the Framingham risk factor set to the Health ABC function. The procedures based on p-values and the AIC lead to very similar final models (Table S2); in contrast, the BIC, which strongly penalizes the complexity of the model, lead to the omission of a larger number of risk factors. All final models mainly retained traditional risk factors included in the FRS. The additions of lifestyle variables (alcohol, physical activity), waist circumference, and creatinine did not improve risk prediction in terms of discrimination or model fit beyond using the traditional risk factors from the FRS. Selection procedures stratified by gender yielded similar results.
In this population-based study of older adults, the FRS poorly discriminated between persons who experienced a CHD event and those who did not (C-index: 0.577 in women; 0.583 in men) and underestimated the absolute CHD risk by 51% in women and 8% in men. Nevertheless, traditional risk factors remained the best predictors of CHD events. Physical activity, alcohol consumption, waist circumference and creatinine did not improve risk prediction beyond traditional risk factors of the FRS. Recalibration of the FRS improved the accuracy of absolute risk estimation, particularly for women. For both genders, the Health ABC function significantly improved estimation of absolute risk, with a discrimation similar to the FRS. Neither refitting equations nor including other routinely available measurements in risk equations provided substantial benefits in terms of discriminating between high- and low-risk older adults over FRS.
Our study adds new data on the performance of recalibration of the FRS, refit functions and the utility of adding other routinely available risk parameters to FRS among older adults. Previous studies also found lower performance of risk prediction based on the FRS associated with increasing age, but did not examine how CHD risk prediction might be improved among older adults. For example, the C-index for the FRS was 0.63/0.66 in men/women aged 65–74 enrolled in the Cardiovascular Health Study  and 0.63 in a patient cohort with a mean age of 66 years , compared to 0.79/0.83 in men/women enrolled in the Framingham Heart Study (mean age of 49 years) . Performance of the FRS may be worse in the very old, with a C-index of 0.53 in adults aged 85 years or older . In different ethnic populations in the US and other countries, FRS often overestimates CHD risk , , . Recalibration of the FRS was shown to improve the estimation of absolute risk in these different ethnic populations , . In the present analysis among older adults, the FRS underestimated absolute CHD risk, particularly in women. Although recalibration of the FRS yielded a better estimation of absolute risk, the function specific to the Health ABC cohort yielded the best estimation of absolute risk, becoming statistically acceptable. Compared to recalibration among other ethnic groups , , the recalibrated FRS showed worse risk prediction in our study of older adults. Our results indicate that the FRS not only underestimates CHD risk in older adults but that some traditional risk factors, such as total and LDL-cholesterol, have weaker associations with CHD risk in older adults, as previoulsy found . In particular, total cholesterol did not predict CHD events in older women in our present study.
Our study has several strengths and limitations. These data are drawn from a well-characterized population-based cohort of older adults, with a high number of CHD events over a 8-year follow-up period, and included a larger sample of black older adults compared to previous studies . CHD events were formally adjudicated. The cohort included both white and black older adults, but did not include other ethnic groups. After stratification by gender, our power for subgroup analyses was limited for comparisons between whites and blacks. Lower performance of the FRS might partly be related to ascertainment of CHD events limited to those requiring hospitalization in the Health ABC, but not in the Framingham cohort . However, all our comparisons in the present data examined CHD outcomes limited to those requiring hospitalization; we also found similar associations for hard CHD events (nonfatal myocardial infarction or coronary death).
What are the potential clinical and research implications of these findings? Clinicians should use the FRS with caution in older adults, as it underestimates the absolute CHD risk by 51% in women and 8% in men and does not discriminate effectively between those who will have CHD events and those who will not. We could not identify additional, routinely available variables that might improve risk prediction beyond traditional risk factors comprising the FRS, similar to several previous studies that did not clearly identify factors improving risk prediction of the FRS . Re-estimated risk functions using these factors improve accurate estimation of absolute risk, but did not meaningfully improve discrimination, or the ability to distinguish between low, intermediate, and high-risk adults. Substantial improvements in discrimination may require novel CHD risk markers or other strategies for risk prediction in the elderly. We have previously found that ankle-arm index and interleukin-6, but not high-sensitive C-reactive protein, improved risk prediction beyond traditional risk factors, but only modestly . Other potential markers that might improve CHD risk prediction in the elderly include homocysteine  or coronary calcification . Future investigations should examine whether markers of atherosclerosis  or novel CHD risk markers  might improve risk prediction beyond FRS in older adults, which still requires additional studies . For current clinical use, recalibrated Framingham functions seem an attractive option to better assess absolute CHD risk for older adults (Methods S1), given that no currently available new risk factors have been clearly and consistently shown to improve CHD risk prediction  and that the Health ABC function needs to be externally validated in another cohort.
In summary, our study suggests that the FRS underestimates CHD risk in the growing population of elderly , particularly in older women. However, traditional risk factors remain the best predictors of future CHD events. Recalibrating risk functions in older adults is important to improve the accuracy of absolute CHD risk estimates, especially for women, and might be useful to better identify older individuals at increased risk who will benefit from preventive therapies, such as statins or aspirin. However, substantial improvements in discrimination may require novel CHD risk markers or other strategies for better CHD risk prediction and risk stratification in the elderly.
Ratio of predicted to observed risks for original Framingham functions (FRS), recalibrated FRS and Health ABC functions across deciles of predicted risk.
Independent predictors of CHD according to different strategies of model selection (n = 2193).
Dr. Rodondi had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis.
Statistical Evaluation. Dr. Vittinghoff, Professor of Biostatistics in the Department of Epidemiology and Biostatistics, University of California, San Francisco, reviewed the statistical analyses of the paper and is included in the authors of this paper.
Conceived and designed the experiments: NR. Performed the experiments: NR IL EV. Analyzed the data: NR IL EV. Contributed reagents/materials/analysis tools: AN DB. Wrote the paper: NR. Critical revision of the manuscript: NR IL DA JB EV ES SS AN PW MP DB. Interpretation of data: NR IL DA JB EV ES SS AN PW MP DB.
- 1. Pletcher MJ, Lazar L, Bibbins-Domingo K, Moran A, Rodondi N, et al. (2009) Comparing impact and cost-effectiveness of primary prevention strategies for lipid-lowering. Ann Intern Med 150: 243–254.
- 2. Wilson PW, D'Agostino RB, Levy D, Belanger AM, Silbershatz H, et al. (1998) Prediction of coronary heart disease using risk factor categories. Circulation 97: 1837–1847.
- 3. Conroy RM, Pyorala K, Fitzgerald AP, Sans S, Menotti A, et al. (2003) Estimation of ten-year risk of fatal cardiovascular disease in Europe: the SCORE project. Eur Heart J 24: 987–1003.
- 4. Assmann G, Cullen P, Schulte H (2002) Simple scoring scheme for calculating the risk of acute coronary events based on the 10-year follow-up of the prospective cardiovascular Munster (PROCAM) study. Circulation 105: 310–315.
- 5. NCEP (2001) Executive Summary of The Third Report of The National Cholesterol Education Program (NCEP) Expert Panel on Detection, Evaluation, And Treatment of High Blood Cholesterol In Adults (Adult Treatment Panel III). JAMA 285: 2486–2497.
- 6. USPSTF (2009) Aspirin for the prevention of cardiovascular disease: U.S. Preventive Services Task Force recommendation statement. Ann Intern Med 150: 396–404.
- 7. Psaty BM, Anderson M, Kronmal RA, Tracy RP, Orchard T, et al. (2004) The association between lipid levels and the risks of incident myocardial infarction, stroke, and total mortality: the cardiovascular health study. J Am Geriatr Soc 52: 1639–1647.
- 8. CDC (2003) Trends in aging–United States and worldwide. MMWR Morb Mortal Wkly Rep 52: 101–104, 106.
- 9. Liu J, Hong Y, D'Agostino RB Sr, Wu Z, Wang W, et al. (2004) Predictive Value for the Chinese Population of the Framingham CHD Risk Assessment Tool Compared With the Chinese Multi-provincial Cohort Study. JAMA 291: 2591–2599.
- 10. Newman AB, Simonsick EM, Naydeck BL, Boudreau RM, Kritchevsky SB, et al. (2006) Association of long-distance corridor walk performance with mortality, cardiovascular disease, mobility limitation, and disability. JAMA 295: 2018–2026.
- 11. Go AS, Chertow GM, Fan D, McCulloch CE, Hsu CY (2004) Chronic kidney disease and the risks of death, cardiovascular events, and hospitalization. N Engl J Med 351: 1296–1305.
- 12. Butler J, Rodondi N, Zhu Y, Figaro K, Fazio S, et al. (2006) Metabolic syndrome and the risk of cardiovascular disease in older adults. J Am Coll Cardiol 47: 1595–1602.
- 13. Maraldi C, Volpato S, Kritchevsky SB, Cesari M, Andresen E, et al. (2006) Impact of inflammation on the relationship among alcohol consumption, mortality, and cardiac events: the health, aging, and body composition study. Arch Intern Med 166: 1490–1497.
- 14. Paffenbarger RS Jr, Hyde RT, Wing AL, Lee IM, Jung DL, et al. (1993) The association of changes in physical-activity level and other lifestyle characteristics with mortality among men. N Engl J Med 328: 538–545.
- 15. Cesari M, Penninx BW, Newman AB, Kritchevsky SB, Nicklas BJ, et al. (2003) Inflammatory markers and onset of cardiovascular events: results from the Health ABC study. Circulation 108: 2317–2322.
- 16. Rodondi N, Newman AB, Vittinghoff E, de Rekeneire N, Satterfield S, et al. (2005) Subclinical hypothyroidism and the risk of heart failure, other cardiovascular events, and death. Arch Intern Med 165: 2460–2466.
- 17. Rodondi N, Marques-Vidal P, Butler J, Sutton-Tyrrell K, Cornuz J, et al. (2010) Markers of atherosclerosis and inflammation for prediction of coronary heart disease in older adults. Am J Epidemiol 171: 540–549.
- 18. Lee CD, Folsom AR, Pankow JS, Brancati FL (2004) Cardiovascular events in diabetic and nondiabetic adults with or without history of myocardial infarction. Circulation 109: 855–860.
- 19. Aktas MK, Ozduran V, Pothier CE, Lang R, Lauer MS (2004) Global risk scores and exercise testing for predicting all-cause mortality in a preventive medicine program. JAMA 292: 1462–1468.
- 20. Harrell FE Jr, Lee KL, Mark DB (1996) Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med 15: 361–387.
- 21. Cook NR, Buring JE, Ridker PM (2006) The Effect of Including C-Reactive Protein in Cardiovascular Risk Prediction Models for Women. Ann Intern Med 145: 21–29.
- 22. Parzen M, Lipsitz SR (1999) A global goodness-of-fit statistic for Cox regression models. Biometrics 55: 580–584.
- 23. Ridker PM, Buring JE, Rifai N, Cook NR (2007) Development and validation of improved algorithms for the assessment of global cardiovascular risk in women: the Reynolds Risk Score. JAMA 297: 611–619.
- 24. D'Agostino RB Sr, Grundy S, Sullivan LM, Wilson P (2001) Validation of the Framingham coronary heart disease prediction scores: results of a multiple ethnic groups investigation. JAMA 286: 180–187.
- 25. Greenland P, LaBree L, Azen SP, Doherty TM, Detrano RC (2004) Coronary artery calcium score combined with Framingham score for risk prediction in asymptomatic individuals. JAMA 291: 210–215.
- 26. de Ruijter W, Westendorp RG, Assendelft WJ, den Elzen WP, de Craen AJ, et al. (2009) Use of Framingham risk score and new biomarkers to predict cardiovascular mortality in older people: population based observational cohort study. BMJ 338: a3083.
- 27. Empana JP, Ducimetiere P, Arveiler D, Ferrieres J, Evans A, et al. (2003) Are the Framingham and PROCAM coronary heart disease risk functions applicable to different European populations? The PRIME Study. Eur Heart J 24: 1903–1911.
- 28. Tzoulaki I, Liberopoulos G, Ioannidis JP (2009) Assessment of claims of improved prediction beyond the Framingham risk score. JAMA 302: 2345–2352.
- 29. Polonsky TS, McClelland RL, Jorgensen NW, Bild DE, Burke GL, et al. (2010) Coronary artery calcium score and risk classification for coronary heart disease prediction. JAMA 303: 1610–1616.
- 30. Gerszten RE, Wang TJ (2008) The search for new cardiovascular biomarkers. Nature 451: 949–952.
- 31. Ioannidis JP, Tzoulaki I (2010) What makes a good predictor?: the evidence applied to coronary artery calcium score. JAMA 303: 1646–1647.
- 32. Levey AS, Coresh J, Greene T, Stevens LA, Zhang YL, et al. (2006) Using standardized serum creatinine values in the modification of diet in renal disease study equation for estimating glomerular filtration rate. Ann Intern Med 145: 247–254.
- 33. Ingelsson E, Schaefer EJ, Contois JH, McNamara JR, Sullivan L, et al. (2007) Clinical utility of different lipid measures for prediction of coronary heart disease in men and women. JAMA 298: 776–785.