Figures
Abstract
Objective
To assess the positive predictive value (PPV) of a clinical score for viral failure among patients fulfilling the WHO-criteria for anti-retroviral treatment (ART) failure in rural Lesotho.
Methods
Patients fulfilling clinical and/or immunological WHO failure-criteria were enrolled. The score includes the following predictors: Prior ART exposure (1 point), CD4-count below baseline (1), 25% and 50% drop from peak CD4-count (1 and 2), hemoglobin drop≥1 g/dL (1), CD4 count<100/µl after 12 months (1), new onset papular pruritic eruption (1), and adherence<95% (3). A nurse assessed the score the day blood was drawn for viral load (VL). Reported confidence intervals (CI) were calculated using Wilsons method.
Results
Among 1'131 patients on ART≥6 months, 134 (11.8%) had immunological and/or clinical failure, 104 (78%) had blood drawn (13 died, 10 lost to follow-up, 7 did not show up). From 92 (88%) a result could be obtained (2 samples hemolysed, 10 lost). Out of these 92 patients 47 (51%) had viral failure (≥5000 copies), 27 (29%) viral suppression (<40) and 18 (20%) intermediate viremia (40–4999). Overall, 20 (22%) had a score≥5. A score≥5 had a PPV of 100% to detect a VL>40 copies (95%CI: 84–100), and of 90% to detect a VL≥5000 copies (70–97). Within the score, adherence<95%, CD4-count<100/µl and papular pruritic eruption were the strongest single predictors. Among 47 patients failing, 8 (17%) died before or within 4 weeks after being switched. Overall mortality was 4 (20%) among those with score≥5 and 4 (5%) if score<5 (OR 4.3; 95%CI: 0.96–18.84, p = 0.057).
Citation: Labhardt ND, Lejone T, Setoko M, Poka M, Ehmer J, Pfeiffer K, et al. (2012) A Clinical Prediction Score in Addition to WHO Criteria for Anti-Retroviral Treatment Failure in Resource-Limited Settings - Experience from Lesotho. PLoS ONE 7(10): e47937. https://doi.org/10.1371/journal.pone.0047937
Editor: Nicolas Sluis-Cremer, University of Pittsburgh, United States of America
Received: July 11, 2012; Accepted: September 25, 2012; Published: October 31, 2012
Copyright: © 2012 Labhardt et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: The authors have no funding or support to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Identifying patients failing on first-line treatment is a major challenge in anti-retroviral treatment (ART) programs in resource-limited settings. As a result patients who are failing on a first-line regimen are often not or not timely switched to a second-line regimen [1]. This leads to a high mortality among patients who fail on first-line ART [1]–[3]. The World Health Organization's (WHO) clinical and immunological failure-criteria showed only limited accuracy. Various studies report a positive predictive value (PPV) of the WHO-criteria for viral failure ranging from 6% to 39% [4], [5]. Thus a confirmatory viral load (VL) measurement is recommended, before any switch to second-line can be considered [5]–[8]. However, in many settings access to viral load testing is difficult and is causing delays in treatment switch.
Lynen et al. published in 2009 a clinical predictor score to identify patients in need for targeted VL testing based on the score's post-test probability for viral failure [9].
The score is based on reported adherence, prior ART exposure, clinical observation of papular pruritic eruption, and trends in CD4-count and haemoglobin-level.
They recommended that only patients with a score 2–4 needed VL testing, as the post-test probability for viral failure in individuals with a score<2 was very low and in case of ≥5 very high. However, the score was derived from a cohort in Cambodia, where only three patients had a score≥5. In a first validation study conducted in Uganda, only 1 patient had a score≥5 [10].
The objective of this study is to assess the PPV of a score≥5 among patients fulfilling the WHO-criteria for treatment failure in a rural cohort of Lesotho, with the aim to reduce the delays in switching to second-line treatment when needed.
Methods
Ethics Statement
The study protocol was approved by the Ethical Committee of the Ministry of Health and Social Welfare of Lesotho. All patients gave oral and written consent. In the case of children <16 years of age, the care-taker gave oral and written consent.
Study objectives and design
The primary objective of this cross-sectional study was to determine the PPV of a clinical score for viral failure among patients on first-line ART who fulfill the clinical and/or immunological WHO-criteria for treatment failure in rural Lesotho. Secondary objectives were to assess the overall failure-rate among patients fulfilling the WHO-criteria, the PPV of each single predictor within the score and the mortality among patients with confirmed viral failure before or within 4 weeks after switch to second-line ART.
Study setting
The study was conducted between October 2010 and April 2011 in the catchment area of Seboche Hospital in northern Lesotho. Seboche Hospital and its 5 affiliated nurse-led health centers serve an estimated catchment population of 55’000. The whole catchment area benefits from support of SolidarMed, a non-governmental organization based in Switzerland who supported the provision of ART since 2005. In October 2010, 1’301 patients were on ART. Of these, 691 (53%) were followed at one of the five health centers, the remaining at the hospital. ART is available free of costs at health facilities in Lesotho. Clinical monitoring is usually done on monthly basis, CD4 counts are determined every 6 months. In case of suspected treatment failure, clinicians have to write a second-line request to a second-line committee headed by the Ministry of Health and Social Welfare of Lesotho. The committee replies within one to two months. Usually, for the request to be approved, confirmatory VL testing is required. At the time the study was conducted, no VL testing was available within Lesotho. However, in case of immunological or clinical failure, the Ministry provides funds for a limited number of VL tests to be performed in a laboratory in South Africa. For this purpose, blood samples have to arrive before 2 pm at the central laboratory in Maseru in order to be further transported to South Africa. All VL obtained in this study followed this procedure.
Study procedure and selection of participants
All patients on first-line ART since at least 6 months, aged ≥10 years, who fulfilled clinical and/or immunological WHO-criteria for treatment failure and who were followed within the study area, were eligible for the study. Clinical failure was defined as a new or recurrent WHO-stage 4 event after at least 6 months of ART. Immunological failure was defined as a drop of the CD4 count back to or below baseline or a ≥50% drop of CD4-count from the on-treatment CD4-peak or persistent CD4 levels <100 cells/µl after ≥12 months on treatment [11]. Even though the second and third immunological WHO-criteria are only applicable for patients on ART since at least 12 months, patients who were on ART for less than 12 months who showed a new clinical stage 4 event after ≥6 months on ART or whose CD4-count fell below pre-treatment levels were included in the study. Patients taking protease-inhibitor based ART were excluded from the study. ART-nurses at hospital and health centers were trained to assess the WHO-criteria for treatment failure and went through all patient-records checking for immunological or clinical signs of failure to generate a list of eligible patients. All eligible patients were scheduled on pre-defined days for blood-draw at Seboche Hospital. Blood-samples were transported the same day to the Central Laboratory of Lesotho at Maseru for further processing. Patients not attending the first scheduled visit were traced and given a second appointment. Table 1 displays the components of the score used in this study. The day blood for VL was drawn, a nurse-clinician assessed the clinical part of the score (adherence measured by a visual analogue scale (VAS), prior exposure to HAART, presence of recurrent papular pruritic eruption). At the same moment blood was drawn for VL, a second tube to repeat CD4-count and hemoglobin was taken. This was done to ensure that CD4-counts and hemoglobin levels used to calculate the score were contemporary, as some patients who were eligible based on the records had their last CD4-count several months prior to the day they came for VL-measurement. All patients who participated in the study and who did not show up for their ART follow-up visits after VL was measured were traced to ascertain if they were alive and to encourage their return back to care.
Data-processing and statistical analysis
All patient-data to build the score were collected the same day the blood sample for VL was taken. Data were entered anonymously using EpiData 3.1® and stored on a password-protected computer. An undetectable VL (<40 copies/µl) was considered as viral suppression, a VL between 40 and 5’000 as indeterminate result and ≥5’000 as viral failure. Based on the clinical score, two subgroups of patients were formed: those with a score<5 and those with a score≥5. The PPV for a VL>40 or a VL≥5’000 copies was calculated as the proportion of individuals with a VL>40 or a VL≥5’000 respectively out of the subgroup of all patients who have a score≥5. Confidence intervals (CI) for the PPV were calculated using Wilsons score method. To assess the risk of mortality among patients continuing first-line ART while waiting for the VL-result and the approval from the MoHSW to be switched to second-line, outcomes of all patients were determined eight weeks after the blood draw for VL. In case of patients with viral failure, their status (dead or alive) was checked the moment they were supposed to start second-line ART (the moment the approval of the MoHSW was received). In patients who were able to start second-line ART at that moment, the status (dead or alive) was again determined 4 weeks after the switch. Risk of mortality during this time-period between blood-draw for VL and 4 weeks after switch to second-line was assessed using univariate logistic regression, with a score≥5 as predictor. All analyses were run on STATA 10.1™.
Results
Participants
Among the 1’301 patients retained and on ART in the study area by October 1st 2010, 1’131 were on ART since at least 6 months, and 134 (11.8%) fulfilled WHO-criteria for immunological and/or clinical failure. Of those, 104 (78%) had blood drawn on one of the scheduled days, 13 had died, 10 were lost to follow-up and 7 did not manage to come at several occasions (see figure 1). From 92 (88%) of the samples a result could be obtained (2 samples hemolysed, 10 were lost during transport/processing in the laboratory). Of the 92 patients enrolled in the study, 75 (82%) had only immunological failure, 1 had only clinical failure and 16 (17%) fulfilled immunological as well as clinical WHO criteria for treatment failure. Most (89) were on ART since at least 12 months. Table 2 shows characteristics of the 92 patients included in the analysis.
92 (69%) of the patients initially eligible for the study received a VL-result and were included in the final analysis. VL: viral load. ART: Anti-retroviral Therapy. WHO: World Health Organization.
From the date of blood-draw to the date the VL-result was obtained, a median time of 28 days (inter-quartile range 25–56) elapsed. Among those patients with viral failure, a median time of 55 days (39–74) elapsed between the day they were scheduled for VL blood-draw and the day a response from the second-line committee was obtained.
Positive Predictive Value of the Clinical Score
Out of the 92 patients with a VL result, 47 (51%) had viral failure (VL≥5000 copies), 27 (29%) viral suppression (<40) and 18 (20%) an intermediate result (40–4999).
Overall, 20 (22%) had a clinical score≥5. A score≥5 had a PPV of 100% to detect a VL>40 copies (95%CI: 84–100), and of 90% to detect a VL≥5000 copies (95%CI: 70–97) (table 3). The 72 patients with a score<5 had still a probability of 40% (95%CI: 30–52) for a VL≥5’000. As shown in table 3, out of the 8 components of the score, an adherence measured by VAS of less than 95%, new onset papular pruritic eruption and a CD4-count <100 cells/µl had the highest positive predictive value as a single predictor in addition to the WHO-criteria. For a detectable VL and viral failure, the PPV of VAS<95% (n = 15) alone was 100% (95%CI: 80–100) and 93% (70–99) respectively, for recurrent papular pruritic eruption (n = 27) it was 89% (72–96) and 85 (68–94) and for a persistent CD4-count below 100cells/µl (n = 27) it was 93% (77–98) and 89% (72–96).
Positive Predictive Value of the WHO-criteria and Mortality
The probability of viral failure (≥5’000 copies) in patients who only had immunological WHO criteria was 44% (95%CI: 33–55) as compared to 81% (95%CI: 57–93) in patients with immunological and clinical WHO criteria combined (table 3). However, patients who had only immunological failure but a score≥5 (n = 13) had viral failure in 92% (95%CI: 67–98) of cases.
Among the 47 patients with viral failure, 8 (17%) died before or within 4 weeks after being switched. Among the patients with VL<5000 all 45 were retained 8 weeks after blood was drawn for VL. The overall mortality among the 92 participants was 4 out of 20 (20%) in the group with score≥5 and 4 out of 72 (5%) among those with a score<5 (OR 4.3; 95%CI: 0.96; 18.84, p = 0.057).
Discussion
This cross-sectional study assessed the PPV of a clinical score for viral failure among patients on first-line ART who fulfilled the immunological and/or clinical WHO-criteria for treatment failure in rural Lesotho. The score applied in this study was originally derived from a cohort in Cambodia with 764 individuals [9]. We found that a score≥5 had a high PPV for viral failure within a subgroup of patients in rural Lesotho who already fulfilled the immunological and/or clinical WHO-criteria. The PPV was 100% for a detectable viral load and 90% for a VL≥5’000 copies, while for the WHO-criteria alone the PPV was 71% and 51%, respectively (table 3). Moreover, patients with a score≥5 tend to have a higher mortality while waiting for, or shortly after, the start of second-line ART. Because of the operational challenges to obtain a viral load in our setting (as highlighted in figure 1), it may be appropriate to switch patients who fulfill the immunological and/or clinical WHO-criteria and have a score≥5 immediately to a second-line regimen without doing a confirmatory VL test. However, given that even those with a score<5 have a reasonable chance of failure (40%) the score cannot be used to exclude viral failure in this subgroup of patients.
This study has several limitations. First, we only measured one viral load per patient. A paper by Castelnuovo et al. showed that a majority of patients who have intermediate or high viremia achieve viral suppression at a second measurement [12]. Saag and colleagues report from a trial in Zambia that nearly half of patients achieved viral suppression at second measurement [13]. The overall failure rate may thus be overestimated in our study. A second limitation is that the applied score still relies on CD4-counts and is therefore not applicable in settings without access to CD4-count measurement. However, Marinucci et al. showed in an assessment in seven African countries that by 2010 already two thirds of rural ART-clinics had access to CD4-count on site [14]. Like the health centers in Lesotho, many other clinics that do not have CD4-counters on site may have a system of regular transport of blood samples to the referral hospital for analysis. A third limitation is that the patient population included may not be fully representative of all patients possibly failing first-line therapy. The study suffers from operational challenges inherent to working in low-resource rural settings. Only 92 (69%) out of 134 eligible patients could be included in the study, as for the others no VL-result could be obtained. Patients have to travel long distances to the health facilities and may be reluctant to come back for a blood test, especially when sick. This demonstrates the need to define criteria that allow switching patients with high probability of viral failure to second-line without prior confirmation through VL testing. Finally, in this rural setting some patients may not have a regularly documented CD4-count while on ART and ART-nurses may have overlooked some clinical WHO stage 4 conditions. Thus the overall number of patients fulfilling clinical and/or immunological WHO failure criteria in the study area may be considerably higher than the 134 patients that were identified during the study period.
Future studies on scoring systems in low-resource settings should look at criteria that do not depend on previous CD4 counts. Our study only looked at the PPV of the score≥5 in addition to WHO criteria. Looking at the individual PPV of the criteria that build the score, the strongest predictors are VAS<95%, a new onset papular pruritic eruption and a CD4-count <100 cells/µl after ≥12 months on ART (Table 3). As in some settings it may be difficult to have all data available to use the score (previous CD4-counts and previous hemoglobin levels), using these criteria alone as additional check before switching patients who fulfill immunological or clinical WHO-criteria to second-line would already increase the PPV for viral failure substantially.
The CD4-cell count alone, as it is used in the WHO definition of immunological failure, has shown limited accuracy in predicting viral failure [15], [16] and has led to misclassification and unnecessary as well as delayed switch to second-line ART [6]–[8], [17]–[19]. Because of the limited accuracy of WHO criteria, confirmation of treatment failure through VL testing has been recommended for all patients [20]. In our study, the PPV of WHO immunological criteria was higher than in previous studies [4], but a PPV of 44% for a VL≥5000 copies (Table 3) is still too low to justify switch to second-line ART without prior confirmation by VL-testing. However, to await a VL result for all patients presenting with failure is difficult in rural settings with limited resources and may result in delayed switching and high mortality. Several studies indicate that use of other laboratory and clinical information, such as adherence or changes in hemoglobin may improve the predictive value [9], [21]. In a South African cohort, Egger and colleagues have recently shown the usefulness of risk-charts that take into account CD4-trajectories over time on ART [22]. However, up to now, none has shown enough accuracy to replace VL testing. As long as cheap and simple to use tests for VL measurements are lacking, there is an urgent need to define clinical criteria that help to narrow down the population that needs VL testing.
Conclusions
A clinical score≥5 in patients fulfilling the clinical and/or immunological WHO treatment failure criteria results in a high probability of viral failure. For these patients, especially in settings with limited access to VL-testing, we may consider switching patients to second-line treatment without prior VL-confirmation. This approach may save money and valuable time for the patient, thereby reducing mortality and lost-to follow-up. However, a lower score cannot be used to exclude viral failure. These patients require a confirmatory VL measurement prior to switch to second-line.
Acknowledgments
The authors would like to thank all patients who participated in the study and the Laboratory team of Seboche Hospital and Lesotho Central Laboratory for transporting and processing the blood-samples. Further the authors would like to thank Dr Joris Menten for critically reviewing the statistical analyses.
Author Contributions
Conceived and designed the experiments: NDL TIL MS MP PZK JE KP. Performed the experiments: NDL TIL MS MP PZK. Analyzed the data: NDL LL. Contributed reagents/materials/analysis tools: NDL MS TIL MP PZK. Wrote the paper: NDL LL JE KP.
References
- 1. Keiser O, Tweya H, Braitstein P, Dabis F, MacPhail P, et al. (2010) Mortality after failure of antiretroviral therapy in sub-Saharan Africa. Trop Med Int Health 15: 251–258.
- 2. Hosseinipour MC, Kumwenda JJ, Weigel R, Brown LB, Mzinganjira D, et al. (2010) Second-line treatment in the Malawi antiretroviral programme: high early mortality, but good outcomes in survivors, despite extensive drug resistance at baseline. HIV Med 11: 510–518.
- 3. Pujades-Rodríguez M, Balkan S, Arnould L, Brinkhof MA, Calmy A (2010) AIDS Working Group of MSF (2010) Treatment failure and mortality factors in patients receiving second-line HIV therapy in resource-limited countries. JAMA 304: 303–312.
- 4. Hosseinipour MC, Schechter M (2010) Monitoring antiretroviral therapy in resource-limited settings: balancing clinical care, technology, and human resources. Curr HIV/AIDS Rep 7: 168–174.
- 5. Rawizza HE, Chaplin B, Meloni ST, Eisen G, Rao T, et al. (2011) Immunologic Criteria Are Poor Predictors of Virologic Outcome: Implications for HIV Treatment Monitoring in Resource-Limited Settings. Clin Infect Dis 53: 1283–1290.
- 6. Kantor R, Diero L, Delong A, Kamle L, Muyonga S, et al. (2009) Misclassification of first-line antiretroviral treatment failure based on immunological monitoring of HIV infection in resource-limited settings. Clin Infect Dis 49: 454–462.
- 7. van Oosterhout JJ, Brown L, Weigel R, Kumwenda JJ, Mzinganjira D, et al. (2009) Diagnosis of antiretroviral therapy failure in Malawi: poor performance of clinical and immunological WHO criteria. Trop Med Int Health 14: 856–861.
- 8. Keiser O, MacPhail P, Boulle A, Wood R, Schechter M, et al. (2009) Accuracy of WHO CD4 cell count criteria for virological failure of antiretroviral therapy. Trop Med Int Health 14: 1220–1225.
- 9. Lynen L, An S, Koole O, Thai S, Ros S, De Munter P, et al. (2009) An algorithm to optimize viral load testing in HIV-positive patients with suspected first-line antiretroviral therapy failure in Cambodia. J Acquir Immune Defic Syndr 52: 40–48.
- 10. Abouyannis M, Menten J, Kiragga A, Lynen L, Robertson G, et al. (2011) Development and validation of systems for rational use of viral load testing in adults receiving first-line ART in sub-Saharan Africa. AIDS 25: 1627–1635.
- 11. World Health Organization (2010) Antiretroviral Therapy for adults and adolescents. Recommendations for a public health approach Available at: www.who.int.
- 12. Castelnuovo B, Sempa J, Agnes KN, Kamya MR, Manabe YC (2011) Evaluation of WHO Criteria for Viral Failure in Patients on Antiretroviral Treatment in Resource-Limited Settings. AIDS Res Treat 2011: 736938.
- 13. Saag M, Westfall A, Luhanga D, Mulenga P, Chi B, et al. (2012) A Cluster Randomized Trial of Routine vs Discretionary Viral Load Monitoring among Adults Starting ART: Zambia. Abstract 87.
- 14. Marinucci F, Medina-Moreno S, Paterniti AD, Wattleworth M, Redfield RR (2011) Decentralization of CD4 testing in resource-limited settings: 7 years of experience in six African countries. Cytometry A 79: 368–374.
- 15. Badri M, Lawn SD, Wood R (2008) Utility of CD4 cell counts for early prediction of virological failure during antiretroviral therapy in a resource-limited setting. BMC Infect Dis 8: 89.
- 16. Bisson GP, Gross R, Strom JB, Rollins C, Bellamy S, et al. (2006) Diagnostic accuracy of CD4 cell count increase for virologic response after initiating highly active antiretroviral therapy. AIDS 20: 1613–1619.
- 17. ART-LINC of IeDEA Study Group (2009) Keiser O, Tweya H, Boulle A, Braitstein P, Schecter M, et al. (2009) Switching to second-line antiretroviral therapy in resource-limited settings: comparison of programmes with and without viral load monitoring. AIDS 23: 1867–1874.
- 18. Levison JH, Orrell C, Losina E, Lu Z, Freedberg KA, et al. (2011) Early outcomes and the virological effect of delayed treatment switching to second-line therapy in an antiretroviral roll-out programme in South Africa. Antivir Ther 16: 853–861.
- 19. Sigaloff KC, Hamers RL, Wallis CL, Kityo C, Siwale M, et al. (2011) Unnecessary antiretroviral treatment switches and accumulation of HIV resistance mutations; two arguments for viral load monitoring in Africa. J Acquir Immune Defic Syndr 58: 23–31.
- 20. Kanapathipillai R, McGuire M, Mogha R, Szumilin E, Heinzelmann A, et al. (2011) Benefit of viral load testing for confirmation of immunological failure in HIV patients treated in rural Malawi. Trop Med Int Health
- 21. Bisson GP, Gross R, Bellamy S, Chittams J, Hislop M, et al. (2008) Pharmacy refill adherence compared with CD4 count changes for monitoring HIV-infected adults on antiretroviral therapy. PLoS Med 5: e109.
- 22. Egger M, Keiser O, Zhou J, Wood R, Garone D, et al. (2012) CD4 Cell Response and Virologic Failure: A Risk Chart. Posterpresentation 634 (accessed March 18th, 2012 at: http://www.retroconference.org/2012b/Abstracts/44171.htm).