Figures
Abstract
Methods
903 HIV positive persons attending field clinics provided a venous blood sample assessed on site using PIMA analyzers per manufacturer's specifications. The venous samples were then run on FACSCalibur flow cytometry at a central facility. The Bland–Altman method was used to estimate mean bias and 95% limits of agreement (LOA). Sensitivity, specificity, negative predictive value (NPV), and positive predictive value (PPV) were calculated for a CD4 threshold of <350 and <500 cells/uL for antiretroviral eligibility.
Results
There was a high correlation between PIMA and FACSCalibur CD4 counts (r = 0.943, p<0.001). Relative to FACSCalibur, the PIMA POC CD4 had negative mean bias of −34.6 cells/uL (95% LOA: −219.8 to 150.6) overall. The dispersion at CD4<350 cells/uL was 5.1 cells/uL (95% LOA: −126.6 to 136.8). For a threshold of CD4<350 cells/uL, PIMA venous blood had a sensitivity of 88.6% (95%CI 84.8–92.4%), specificity of 87.5% (95%CI 84.9–90.1%), NPV of 94.9% (95%CI 93.1–96.7%), and PPV of 74.4% (95%CI 69.6–79.2%). PIMA sensitivity and PPV significantly increased to 96.1% and 88.3% respectively with increased threshold of 500 cells/uL.
Citation: Galiwango RM, Lubyayi L, Musoke R, Kalibbala S, Buwembo M, Kasule J, et al. (2014) Field Evaluation of PIMA Point-of-Care CD4 Testing in Rakai, Uganda. PLoS ONE 9(3): e88928. https://doi.org/10.1371/journal.pone.0088928
Editor: Omar Sued, Fundacion Huesped, Argentina
Received: December 5, 2012; Accepted: January 16, 2014; Published: March 10, 2014
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: We acknowledge CDC Uganda for the funding support that availed the PIMAs to our institution and PEPFAR Uganda that supports the treatment program. This work was supported in part (SJR) by the Division of Intramural Research, National Institute of Allergy and Infectious Diseases, National Institutes of Health. The funders had no role in the study design, data collection, analysis, decision to publish or preparation of the manuscript.
Competing interests: RHG is a PLOS ONE editorial board member. This does not alter our adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
Assessment of eligibility for antiretroviral therapy (ART) in low and middle-income countries (LMIC) is dependent on CD4 T-lymphocyte enumeration and, if unavailable, WHO clinical staging. [1], [2]. Additionally, CD4 testing is frequently used to monitor patients on ART. In most clinical settings, CD4 enumeration has typically used flow cytometry which has excellent precision, accuracy, and reproducibility. However, flow cytometry requires fresh venous blood and subsequent processing is often at a central facility distant from the sample collection point. Consequently, there is a significant time lapse from sample procurement to return of results, with delays in decisions on ART eligibility and initiation. These delays are a barrier to care and can result in losses to follow-up with negative patient and programmatic outcomes [3]–[6].
Point-of-care (POC) CD4 T-lymphocyte enumeration is a promising technology that could mitigate many of the barriers and delays associated with flow cytometry and improve access to care. POC CD4 testing could allow timely return of CD4 results to providers and patients, obviating the need for repeated patient visits and rapidly linking patients into care. Several evaluations of POC CD4 technology have been reported, but results have not been entirely consistent, and study settings have been largely confined to central laboratories in urban locations [7]–[13]. Important questions remain about how this technology compares to established methods in primary healthcare settings. We conducted a field evaluation of the PIMA POC CD4 test system (Alere Inc., Waltham, MA) in a rural setting in Rakai, Uganda.
Materials and Methods
Study Population
Rakai district in southwestern Uganda has a population of approximately half a million in a rural area of about 5,000 square kilometers. The Rakai Health Sciences Program (RHSP) is a community-based research organization with a focus on HIV/AIDS and reproductive health [14] which since 2004 has been a major PEPFAR implementer. In March of 2012, RHSP introduced the PIMA POC CD4 platform into the clinical program to expedite assessment of ART eligibility and monitoring of care. This evaluation consists of POC testing done until May 2012 among HIV-infected patients who were both ART naïve (pre-ART) and ART experienced persons. No informed consent was required as data was originally collected for clinical purposes and was analyzed anonymously.
The study was approved by institutional review boards at the Uganda Virus Research Institute's Safety and Ethics Committee, the Uganda National Council for Science and Technology, and Johns Hopkins University, School of Medicine.
POC CD4 Testing
POC CD4 testing was performed with the PIMA CD4 analyzer. All testing followed manufacturer's instructions and involved use of four PIMA machines. The machines were located at a testing area next to the clinics and the venous samples were collected by the nursing team. Briefly, prior to daily testing, normal and low controls were run on the PIMA analyzer. Test cartridges were inspected for expiry dates, assembled, and participant identifiers entered [9], [15]. All testing was carried out by qualified laboratory technicians who had received training in PIMA usage. A result print out was available via an external PIMA printer for clinical review. At the end of each day, an electronic record of all tests identifiers and results was extracted from the PIMA and entered into a clinical cohort database.
PIMA testing was conducted on venous blood collected in a sterile 4 mL EDTA tube and inverted 8–10 times. All samples analyzed on PIMA were stored at ambient temperature (18–25°C) and run within 8 hours of collection. The tube was further inverted 10–15 times to ensure adequate sample mixing prior to volumetric pipette transfer to fill the sample collector of the PIMA CD4 test cartridge. The remaining blood was stored in cool boxes and dispatched to the laboratory for flow cytometry.
FACSCalibur Testing
CD4 flow cytometry was performed using FACSCalibur (Becton Dickinson, San Jose, California, USA). CD4 estimation was done per manufacturer's instructions by trained laboratory technicians. The FACSCaliburs are regularly serviced and run external quality proficiency samples from UKNEQAS leukocyte immunophenotyping and College of American Pathologists (CAP). Samples were kept at ambient temperature (20–25°C) and run on FACSCalibur within 24 hours of blood collection. Daily instrument calibration was done using calibrate beads to adjust settings, set fluorescence compensation and ensure instrument sensitivity. Sample preparation was done by adding 20 uL of Tritest CD3, 4, 45 antibodies to a patient labeled truecount tube and 50 uL of whole blood prior to 15 minutes dark room incubation after which samples were loaded and analyzed. Work lists were generated, saved and results examined with manual gating done to fine adjust the CD4population before review by the laboratory manager. Data was exported and merged with the clinical cohort database.
Statistical Methods
Descriptive statistics were used to summarize patient characteristics. Accuracy of CD4 results (FACSCalibur versus PIMA) were compared using the Bland-Altman Method which assesses bias as measured by the average difference and calculated 95% limits of agreement (LOA) of all the paired measurements [16]. Paired t tests were done to compare differences in means. Two by two tables were constructed to assess the diagnostic accuracy at CD4 thresholds of <350 and <500 cells/uL thresholds. The <350 threshold is the current Ugandan criteria for ART initiation and the 500 threshold is the new WHO recommended criteria for initiation of ART. All data were analyzed with STATA 12.1 software (College Station, Texas, USA).
Results
A total of 903 HIV-positive patients contributed data for this evaluation and 258 (28.5%) were on ART (Table 1). Scatter plots showed high correlation (r = 0.943, p<0.001) between PIMA and FACSCalibur estimates (Figure 1).
Paired PIMA and FACSCalibur results were compared using Bland-Altman analysis (Fig. 2). The mean bias for the PIMA testing was −34.6 cells/µl (95% LOA: −219.8 to 150.6. Table 2). The overall mean CD4 count with PIMA was significantly lower than with FACSCalibur (p<0.0001), primarily due to underestimation of CD4 counts at higher CD4 levels ≥350 cells/uL (p<0.0001).
The diagnostic accuracy of PIMA POC CD4 measurements was compared to FACSCalibur as the <350 cells/uL threshold for determining ART eligibility (Table 3). The PIMA sensitivity was 88.6% (95% CI 84.8–92.4%) and specificity was 87.5% (95% CI 84.9–90.1%).
We assessed the mean CD4 counts among patients who would be inaccurately classified as ART ineligible (i.e., false negative PIMA CD4 estimates) to better understand the likelihood of being found to be ART eligible at a subsequent test (Table 4). Samples misclassified by PIMA as above this level (350 cells/uL), had a PIMA mean CD4 count of 401 cells/uL which was significantly higher than the 350 cells/uL cut off (p<0.001). Similarly, samples misclassified by PIMA as false positive had a mean count of 307 cells/uL which was again significantly lower than the threshold of 350 cell/uL (p<0.001).
Increasing the threshold 500 cells/uL significantly increased the PIMA sensitivity to 96.1% and PPV to 88.3%. However, specificity and NPV declined though not significantly (Tables 5 and 6). The PIMA mean CD4 count (566.2 cells/uL) of samples misclassified as negative (n = 20) was significantly higher than the cutoff of 500 cells/uL (p = 0.0001). Similarly, sample misclassified by PIMA as positive had an average PIMA CD4 count of 425 cells/µl which was significantly lower than the threshold of 500 cells/µl (p<0.0001, Table 7). At thresholds of 350 and 500, PIMA validity was higher in patients not on ART compared to patients on ART, though the differences were not statistically significant (results not shown).
Discussion
Delays in CD4 testing for assessment of ART eligibility is a significant impediment to ART rollout in LMIC. By reducing delays, POC technologies for CD4 estimation could help linkage into HIV care. We found that PIMA measurements compared to FACSCalibur produced negatively biased CD4 estimates which were more pronounced at CD4 counts >350 cells/µl. In our rural field setting, PIMA POC CD4 using venous blood demonstrated lower sensitivity at identifying persons eligible for ART at a CD4 count of 350 cells/µl. For the 11.4% of patients testing not eligible for ART by PIMA but eligible by FACSCalibur (i.e., false negatives), the mean PIMA estimates were significantly greater than 350 cells/uL, suggesting this patient population would likely benefit from early repeat testing. This finding is similar to a Kenyan study [17]. At a CD4 cutoff of 500 cells/uL, PIMA sensitivity increased significantly and this may support use of this technology when treatment guidelines recommend ART initiation at 500 cells/uL. Other field based studies are however necessary to establish the consistency of this finding.
POC CD4 testing with PIMA enables same day, on site immunological assessment of HIV positive clients due to its portability, printer options, limited blood volume requirements, and ease of use that eliminates initial blood processing and reagent preparation. In our setting, with a 20 minute turnaround time for each sample, busy clinics required two to four PIMA instruments and an additional technician to complete patient testing on any given day. This adds to cost in resource limited settings. A drawback identified by previous studies include inability of PIMA to predict CD4% for evaluations in children aged 2–6 years [12]. Our results showed increasing dispersion with CD4 counts >350 cells/uL which may reflect a proportional bias with increasing CD4 counts. It is noteworthy that in our study setting, the machines were moved to the testing site from the central laboratory everyday and this may have affected sensitivity and specificity for venous testing compared to a field evaluation in India [18]. The study in India was a multi-center study and each center had a PIMA instrument and a reference standard machine on site. In spite of this possibility, it is not known what an acceptable limit of misclassification should be and may well vary from program to program [12].
Limitations to this study include the fact that several technicians participated and there are possible inter-observer variations in machine handling. We also did not assess finger prick testing sampling in our study which may by an attractive option for some programs. Finally, our study was conducted at a single institution and results may not be generalizable to other settings.
Given the scarcity of FACSCalibur testing in many LMIC, POC technologies could facilitate screening for ART eligibility, but continued validation of POC CD4 platforms will be needed with scale-up in diverse situations.
Acknowledgments
We would like to thank Brian Mpoza and Viola Ssekweyama for their technical assistance and Mr. Rob Burris for the training in PIMA POCs using venous blood.
Author Contributions
Conceived and designed the experiments: RMG RHG. Performed the experiments: MB JK SK. Analyzed the data: LL RM RHG SJR LWC. Wrote the paper: RMG RHG RM SJR LWC DS.
References
- 1. Kagaayi J, Makumbi F, Nakigozi G, Wawer MJ, Gray RH, et al. (2007) WHO HIV clinical staging or CD4 cell counts for antiretroviral therapy eligibility assessment? An evaluation in rural Rakai district, Uganda. AIDS 21 (9) 1208–10.
- 2. Jaffar S, Birungi J, Grosskurth H, Amuron B, Namara G, et al. (2008) Use of WHO clinical stage for assessing patient eligibility to antiretroviral therapy in a routine health service setting in Jinja, Uganda. AIDS Res Ther 5: 4.
- 3. Lawn SD, Myer L, Orrell C, Bekker LG, Wood R, et al. (2005) Early mortality among adults accessing a community-based antiretroviral service in South Africa: implications for programme design. AIDS 19 (18) 2141–8.
- 4. Lawn SD, Myer L, Harling G, Orrell C, Bekker LG, et al. (2006) Determinants of mortality and nondeath losses from an antiretroviral treatment service in South Africa: implications for program evaluation. Clin Infect Dis 43 (6) 770–6.
- 5. Brinkhof MW, Boulle A, Weigel R, Messou E, Mathers C, et al. (2009) Mortality of HIV-infected patients starting antiretroviral therapy in sub-Saharan Africa: comparison with HIV-unrelated mortality. PLoS Med 6 (4) e1000066.
- 6. Stringer JS, Zulu I, Levy J, Stringer EM, Mwango A, et al. (2006) Rapid scale-up of antiretroviral therapy at primary care sites in Zambia: feasibility and early outcomes. JAMA 296 (7) 782–93.
- 7. Manabe YC, Wang Y, Elbireer A, Auerbach B, Castelnuovo B (2012) Evaluation of portable point-of-care CD4 counter with high sensitivity for detecting patients eligible for antiretroviral therapy. PLoS One 7 (4) e34319.
- 8. Mnyani CN, McIntyre JA, Myer L (2012) The reliability of point-of-care CD4 testing in identifying HIV-infected pregnant women eligible for antiretroviral therapy. J Acquir Immune Defic Syndr 60 (3) 260–4.
- 9. Diaw PA, Daneau G, Coly AA, Ndiaye BP, Wade D, et al. Multisite evaluation of a point-of-care instrument for CD4(+) T-cell enumeration using venous and finger-prick blood: the PIMA CD4. J Acquir Immune Defic Syndr 58 (4) e103–11.
- 10. Herbert S, Edwards S, Carrick G, Copas A, Sandford C, et al. (2012) Evaluation of PIMA point-of-care CD4 testing in a large UK HIV service. Sex Transm Infect 88 (6) 413–7.
- 11. Jani IV, Sitoe NE, Chongo PL, Alfai ER, Quevedo JI, et al. (2011) Accurate CD4 T-cell enumeration and antiretroviral drug toxicity monitoring in primary healthcare clinics using point-of-care testing. AIDS 25 (6) 807–12.
- 12. Mtapuri-Zinyowera S, Chideme M, Mangwanya D, Mugurungi O, Gudukeya S, et al. (2010) Evaluation of the PIMA point-of-care CD4 analyzer in VCT clinics in Zimbabwe. J Acquir Immune Defic Syndr 55 (1) 1–7.
- 13. Glencross DK, Coetzee LM, Faal M, Masango M, Stevens WS, et al. (2012) Performance evaluation of the Pima point-of-care CD4 analyser using capillary blood sampling in field tests in South Africa. J Int AIDS Soc 15 (1) 3.
- 14. Kiwanuka N, Robb M, Kigozi G, Birx D, Philips J, et al. (2004) Knowledge about vaccines and willingness to participate in preventive HIV vaccine trials: a population-based study, Rakai, Uganda. J Acquir Immune Defic Syndr 36 (2) 721–5.
- 15.
Alere website, Alere Pima CD4 Customer Training - http://www.alere.co.uk/sexual-health/articles/alere-pimatm-cd4-customer-training-241/text.htm, Alere.Accessed 2013 October 12.
- 16. Bland JM, Altman DG (1986) Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 1 (8476) 307–10.
- 17. Mwau M, Adungo F, Kadima S, Njagi E, Kirwaye C, et al. (2013) Evaluation of PIMA(R) point of care technology for CD4 T cell enumeration in Kenya. PLoS One 8 (6) e67612.
- 18. Thakar M, Mahajan B, Shaikh N, Bagwan S, Sane S, et al. (2012) Utility of the point of care CD4 analyzer, PIMA, to enumerate CD4 counts in the field settings in India. AIDS Res Ther 9 (1) 26.