Advertisement
Research Article

Performance of a Limiting-Antigen Avidity Enzyme Immunoassay for Cross-Sectional Estimation of HIV Incidence in the United States

  • Jacob Konikoff,

    Affiliation: Department of Biostatistics, School of Public Health, University of California Los Angeles, Los Angeles, California, United States of America

    X
  • Ron Brookmeyer,

    Affiliation: Department of Biostatistics, School of Public Health, University of California Los Angeles, Los Angeles, California, United States of America

    X
  • Andrew F. Longosz,

    Affiliation: National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America

    X
  • Matthew M. Cousins,

    Affiliation: Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

    X
  • Connie Celum,

    Affiliation: Departments. of Global Health and Medicine, University of Washington, Seattle, Washington, United States of America

    X
  • Susan P. Buchbinder,

    Affiliation: Bridge HIV, San Francisco Department of Health, San Francisco, California, United States of America

    X
  • George R. Seage III,

    Affiliation: Department of Epidemiology, Harvard School of Public Health, Boston, Massachusetts, United States of America

    X
  • Gregory D. Kirk,

    Affiliation: Department of Epidemiology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

    X
  • Richard D. Moore,

    Affiliation: Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

    X
  • Shruti H. Mehta,

    Affiliation: Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

    X
  • Joseph B. Margolick,

    Affiliation: Department of Molecular Microbiology and Immunology, Johns Hopkins Bloomberg School of Public Health, Baltimore, Maryland, United States of America

    X
  • Joelle Brown,

    Affiliation: Department of Epidemiology, School of Public Health, University of California Los Angeles, Los Angeles, California, United States of America

    X
  • Kenneth H. Mayer,

    Affiliation: Department of Global Health and Population Harvard School of Public Health, Boston, Massachusetts, United States of America

    X
  • Beryl A. Koblin,

    Affiliation: New York Blood Center, New York, New York, United States of America

    X
  • Jessica E. Justman,

    Affiliation: Department of Epidemiology, Columbia University Mailman School of Public Health, New York, United States of America

    X
  • Sally L. Hodder,

    Affiliation: Department of Medicine, New Jersey Medical School, Newark, New Jersey, United States of America

    X
  • Thomas C. Quinn,

    Affiliation: National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America

    X
  • Susan H. Eshleman,

    Affiliation: Department of Pathology, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

    X
  • Oliver Laeyendecker mail

    olaeyen1@jhmi.edu

    Affiliations: National Institute of Allergy and Infectious Diseases, National Institutes of Health, Bethesda, Maryland, United States of America, Department of Medicine, Johns Hopkins University School of Medicine, Baltimore, Maryland, United States of America

    X
  • Published: December 27, 2013
  • DOI: 10.1371/journal.pone.0082772

Abstract

Background

A limiting antigen avidity enzyme immunoassay (HIV-1 LAg-Avidity assay) was recently developed for cross-sectional HIV incidence estimation. We evaluated the performance of the LAg-Avidity assay alone and in multi-assay algorithms (MAAs) that included other biomarkers.

Methods and Findings

Performance of testing algorithms was evaluated using 2,282 samples from individuals in the United States collected 1 month to >8 years after HIV seroconversion. The capacity of selected testing algorithms to accurately estimate incidence was evaluated in three longitudinal cohorts. When used in a single-assay format, the LAg-Avidity assay classified some individuals infected >5 years as assay positive and failed to provide reliable incidence estimates in cohorts that included individuals with long-term infections. We evaluated >500,000 testing algorithms, that included the LAg-Avidity assay alone and MAAs with other biomarkers (BED capture immunoassay [BED-CEIA], BioRad-Avidity assay, HIV viral load, CD4 cell count), varying the assays and assay cutoffs. We identified an optimized 2-assay MAA that included the LAg-Avidity and BioRad-Avidity assays, and an optimized 4-assay MAA that included those assays, as well as HIV viral load and CD4 cell count. The two optimized MAAs classified all 845 samples from individuals infected >5 years as MAA negative and estimated incidence within a year of sample collection. These two MAAs produced incidence estimates that were consistent with those from longitudinal follow-up of cohorts. A comparison of the laboratory assay costs of the MAAs was also performed, and we found that the costs associated with the optimal two assay MAA were substantially less than with the four assay MAA.

Conclusions

The LAg-Avidity assay did not perform well in a single-assay format, regardless of the assay cutoff. MAAs that include the LAg-Avidity and BioRad-Avidity assays, with or without viral load and CD4 cell count, provide accurate incidence estimates.

Introduction

HIV incidence measures the rate of new HIV infections in a population [1]. Accurate incidence estimates are needed to identify populations at increased risk of HIV acquisition, monitor the HIV/AIDS epidemic, and evaluate interventions for HIV prevention [2], [3]. Cohort studies that identify HIV seroconverters are expensive and time-consuming, and may not provide reliable incidence estimates because of selection bias, changes in behavior associated with study participation, and loss to follow-up [4]. Unfortunately, serologic assays developed for cross-sectional incidence estimation often overestimate HIV incidence because some long-term infections are classified as assay positive (incident) [5][9]. Use of multi-assay algorithms (MAAs) to estimate incidence in cross-sectional surveys is a promising alternative approach for estimating HIV incidence [10], [11]. We recently demonstrated that accurate incidence estimates can be obtained using MAAs that include the BED capture immunoassay (BED-CEIA) [12], the BioRad-Avidity assay [13], and viral load, with or without a second non-serologic biomarker (either CD4 cell count or HIV diversity) [14][16].

The United States (US) Centers for Disease Control recently developed a limiting antigen avidity enzyme immunoassay (LAg-Avidity assay) for cross-sectional HIV incidence estimation [17], [18]. This assay measures the avidity of antibody binding to low concentrations of a multi-subtype peptide derived from an immunodominant region of gp41. This assay is commercially available and has been promoted for HIV incidence estimation [19], [20]. However, data from other research groups supporting use of this assay are limited [21], and a recent editorial called for a thorough independent evaluation of the LAg-Avidity assay [22]. The kit manufacturer (SEDIA Biosciences, Portland, OR) recommends excluding individuals with advanced HIV disease (e.g., CD4 cell count <200 cells/mm3), elite controllers, and individuals on antiretroviral therapy from incidence surveys [23]. However, no guidance is provided in the package insert on how to combine data from the LAg-Avidity assay and other assays, or what cutoffs should be used for other assays (e.g., HIV viral load). The manufacturer recently recommended increasing the LAg-Avidity assay cutoff from 1.0 to 1.5 normalized optical density units (OD-n) and reducing the mean window period used for incidence estimation from 141 to 130 days [24]; the mean window period is the average time that an individual appears recently infected using an assay or MAA.

In this report, we evaluated the performance of the LAg-Avidity assay alone and in MAAs that included other serologic and non-serologic biomarkers. We evaluated the probability that samples were classified as assay positive or MAA positive over time (using tabulated classification frequencies and fitted smoothed spline curves). We also compared the mean window periods and shadows of the different testing approaches, as well as the relative sample sizes needed for incidence surveys, and relative costs. Finally, we evaluated the ability of the different testing approaches to produce incidence estimates in three cohorts with low, medium, and high incidence that were consistent with those obtained by direct longitudinal follow-up.

Methods

Ethics Statement

Written informed consent was obtained from study participants and all studies were reviewed and approved by relevant institutional review boards. Only stored samples from individuals who consented to have their samples could be used for future research were used in this investigation. No new samples were obtained specifically for this study. The study for cross sectional incidence testing on stored study samples was approved by the institutional review board of the Johns Hopkins University. The research was conducted according to the principles expressed in the Declaration of Helsinki.

Samples used for analysis

The performance of the LAg-Avidity assay and MAAs that include the LAg-Avidity assay was evaluated using 1,782 plasma and serum samples from three cohort studies in the US (duration of infection from 1 month to >8 years): the Multicenter AIDS Cohort Study [25] (MACS, men who have sex with men [MSM], 564 samples, 3 to >8 years after seroconversion), the AIDS Linked to the IntraVenous Experience cohort [26] (ALIVE, persons who inject drugs, 410 samples, 2 to 6 years after seroconversion), and the HIV Network for Prevention Trials 001/001.1 vaccine preparedness cohort [27] (HIVNET 001, men and women with different risk factors for HIV infection, 808 samples, 1 month to 4 years after seroconversion). Further assessments were performed using 500 additional samples from the Johns Hopkins Hospital Clinical Cohort [28] (JHHCC, varied risk factors, approximately one half are persons who inject drugs, all >8 years after seroconversion). Detailed descriptions of these sample sets are provided in previous publications [14], [29]. Two of the 1,782 samples did not have data from the LAg-Avidity assay; those samples were not included in the initial search for optimal testing algorithms (see below), but were included in some of the subsequent analyses. Cross-sectional incidence estimates were generated using additional samples obtained from the HIVNET 001 study (see above) [27], and two other cohort studies from the US: the HIV Prevention Trials Network (HPTN) 061 study (a study of black MSM in the US) [30] and the HPTN 064 study (a study of women with increased HIV acquisition risk from high poverty and HIV prevalence areas in the US) [31].

Laboratory methods

The LAg-Avidity assay (HIV-1 LAg-Avidity EIA, SEDIA Biosciences Corporation, Portland, OR) was performed according to the manufacturer's instructions. The results are reported as OD-n. Samples with an initial result <2.0 OD-n were retested in triplicate for confirmation and the median confirmation value was used for analysis. Most of the data for the BED-CEIA, the BioRad-Avidity assay, and HIV viral load were generated in previous studies. The BED-CEIA (Calypte Biomedical Corporation, Lake Oswego, OR) was performed according to the manufacturer's instructions [12]. The BioRad-Avidity assay is based on the Genetic Systems 1/2+O ELISA [13] (Bio-Rad Laboratories, Redmond, WA), using diethyl amine as the chaotropic agent with the following modification: the diethyl amine was diluted in water and the initial incubation time was decreased to 30 minutes. CD4 cell count data were obtained in the cohort studies.

Statistical Methods

Samples analyzed using the LAg-Avidity assay alone were classified as assay positive (below the assay cutoff) or assay negative (above the assay cutoff); similarly, samples analyzed using MAAs were classified as MAA positive (meeting the criteria of all component assays) or MAA negative (failing to meet the criteria of one or more of the component assays). Samples classified as assay positive or MAA positive are counted as incident infections in incidence estimates.

In the first phase of the analysis, we calculated the number of samples classified as assay positive or MAA positive by duration of infection (years after HIV seroconversion). For individuals who had acute or early HIV infection at the time of sample collection (Fiebig stage I or II [32]), seroconversion was assumed to have occurred within 28 days after sample collection. For other individuals, seroconversion was assumed to have occurred between the last negative and first positive HIV tests. For these individuals the median number of days between last negative and first positive HIV tests was 184 days (interquartile range: 168–201 days). We first assessed performance of: (1) the LAg-Avidity assay alone (assay cutoffs: 0.5, 1.0, 1.5, or 3.0 OD-n), (2) a 3-assay MAA that included the LAg-Avidity assay (assay cutoff: 1.5 OD-n) with viral load (assay cutoff: 1,000 copies/mL) and CD4 cell count (assay cutoff: 200 cells/mm3), and (3) 2-assay MAAs that included the LAg-Avidity assay (assay cutoffs of 1.0 or 1.5 OD-n) and viral load (assay cutoff: 1,000 copies/mL) without CD4 cell count. For these analyses, duration of infection was classified into time intervals by midpoint imputation.

The proportion of samples that were classified as assay positive or MAA positive was also assessed as a function of duration of infection by fitting logistic regression models with cubic splines [14]. Seroconversion dates were sampled from uniform distributions over the potential seroconversion intervals defined above. The mean window period (i.e., mean duration a person is assay or MAA positive) and the shadow (a measure of how far back in time incidence is being estimated [32], [33]) were estimated by using multiple imputation and averaging the results from the fitted curves [14], [33], [34]. The mean window period and shadow were not calculated for an assay or MAA if the probability of a positive result did not converge to zero (<0.001) by 8 years after seroconversion. To account for multiple samples from the same individual, confidence intervals (CIs) were calculated using bootstrapping, blocking on individuals (so that all samples from the same individual were included in each bootstrapped sample). The bootstrap was stratified by cohort study [14], [33], [34].

The second phase of the analysis further assessed the performance of the LAg-Avidity assay alone and MAAs that included other serologic and non-serologic assays to determine optimal algorithms based on criteria described below. Over 30,000 testing algorithms were considered by analyzing all possible combinations of the following cutoffs for the component assays: LAg-Avidity assay: 22 cutoffs ranging from 0.5–3.9 OD-n; BioRad-Avidity assay: 12 cutoffs ranging from 30–100% (avidity index); viral load: 10 cutoffs ranging from 400–10,000 copies/mL; CD4 cell count: 12 cutoffs ranging from 50–1,000 cells/mm3. The goal of this analysis was to evaluate different cutoffs for the LAg-Avidity assay alone, and to identify MAAs with optimal assay combinations and assay cutoffs [14]. Testing algorithms were evaluated further if they met the following requirements: (1) the estimated probability of being classified as assay or MAA positive 8 years after seroconversion was <0.001 (based on analysis of samples from the MACS, ALIVE, and HIVNET 001 cohorts), (2) all 500 samples from the JHHCC (all infected >8 years) were classified as assay or MAA negative, (3) the shadow was <250 days and the upper 95% confidence limit of the shadow was <1 year. For each class of testing algorithms that met these requirements, we identified and deemed optimal the assay or MAA that had the largest mean window period, as this minimizes the variability in incidence estimates [33]. We then expanded the search to >500,000 algorithms that included all combinations of the assays and cutoffs listed above, along with the BED-CEIA (using 16 cutoffs for the BED-CEIA ranging from 0.4 to 1.8 OD-n).

In the third phase of the analysis, we compared the number of samples needed in cross-sectional surveys to obtain the same precision in incidence estimates using two optimized MAAs (relative sample size, calculated as the ratio of the mean window periods). We also compared the relative cost of the two MAAs, optimizing the order of the component assays. For this analysis, the relative costs of the LAg-Avidity assay, BioRad-Avidity assay, CD4 cell count, and viral load were considered to be r, r, 2.5r, and 5r, respectively, where r is the unit cost of the LAg-Avidity assay. For the cost analysis, CD4 cell count was listed first when used in a MAA, since CD4 cell count must be performed at the time of sample collection. In addition, costs were adjusted by multiplying the relative costs by the relative sample sizes that are needed to account for differences in mean window periods. These analyses considered only the cost of sample testing.

In the last phase of the analysis, we compared cross-sectional incidence estimates obtained using the LAg-Avidity assay and MAAs to longitudinal incidence estimates previously reported for three cohort studies from the US with varying HIV incidence. Cross-sectional incidence estimates were calculated for: (1) the LAg-Avidity assay alone (using assay cutoffs of 1.0 or 1.5 OD-n), (2) the LAg-Avidity assay with viral load and CD4 cell count (using assay cutoffs of 1.5 OD-n, 1,000 copies/mL, and 200 cells/mm3, respectively), and (3) two optimized MAAs. These analyses were performed for the following three cohorts: HIVNET 001 (see above), HPTN 061 (a cohort of black MSM) [30], and HPTN 064 (a cohort of women in high poverty areas at increase risk of HIV acquisition) [31]. For these analyses, cross-sectional incidence estimates were calculated as: [(number of samples classified as assay or MAA positive)×(100)]/[(number of HIV-negative samples)×(mean window period in years)]. The CIs calculated for these estimates accounted for uncertainty in the mean window period [35].

Human subjects

The cohort studies described in this report were conducted according to the ethical standards set forth by the institutional review boards of the participating institutions and the Helsinki Declaration of the World Medical Association; participants provided written informed consent. The work described in this report involved analysis of stored samples and data from those studies. No participants were recruited or followed in the course of this work. This research was approved by the Institutional Review Board of the Johns Hopkins University.

Results

Performance of the LAg-Avidity assay used in a single-assay format

We first calculated the proportion of samples that were classified as assay positive by duration of infection using the Lag-Avidity assay alone, using four different assay cutoffs (Table 1). Using the cutoff of 1.0 OD-n that was originally recommended for this assay [23], 32% of samples from individuals infected <6 months were classified as assay positive, while 3% of samples from individuals infected >5 years were classified as assay positive. Raising the assay cutoff to 1.5 OD-n (the cutoff currently recommended by the manufacturer [24]) increased the percentage of samples from individuals infected <6 months that were classified as assay positive (from 32% to 43%), but also increased the percentage of samples from individuals infected >5 years who were classified as assay positive (from 3% to 5%). When the assay cutoff was increased to 3.0 OD-n, a greater proportion of samples from individuals infected <6 months were classified as assay positive (74%), but a much larger proportion of samples from individuals infected >5 years were classified as assay positive (15%). When the assay cutoff was lowered to 0.5 OD-n, the percentage of individuals infected >5 years classified as assay positive dropped to 2%; however, at this stringent cutoff, only 13% of samples from individuals infected <6 months were classified as MAA positive.

thumbnail

Table 1. Number of samples classified as assay positive using the LAg-Avidity assay alone.

doi:10.1371/journal.pone.0082772.t001

The proportion of individuals classified as assay positive using the LAg-Avidity assay alone was also analyzed by fitting probability models with cubic splines (Figure 1A). These probability models did not approach 0% assay positive by 8 years using any of the four cutoffs; this indicates that the assay continued to classify some individuals as assay positive 8 years after seroconversion. We did not calculate mean window periods or shadows for the LAg-Avidity assay using any of these assay cutoffs, because of the persistent classification of samples from individuals with long-term HIV infection as assay positive.

thumbnail

Figure 1. Proportion of samples classified as assay positive using the LAg-Avidity assay alone or with HIV viral load, as a function of the duration of HIV infection.

Probability curves were generated by analyzing samples from three cohort studies (see Methods). (A) Probability curves generated using the LAg-Avidity assay with four different assay cutoffs (0.5, 1.0, 1.5, and 3.0 normalized optical density units [OD-n]). Samples were classified as assay positive if the LAg-Avidity assay result was below the assay cutoff. (B) Probability curves generated using the MAAs shown in Figure 2. Samples were classified as MAA positive if results from each of the component assays met the requirements of the MAA.

doi:10.1371/journal.pone.0082772.g001

Performance of the LAg-Avidity assay in MAAs that included HIV viral load, with or without CD4 cell count

The package insert included with the LAg-Avidity assay recommends excluding individuals with advanced HIV disease (CD4 cell count <200 cells/mm3), elite controllers, and individuals on antiretroviral therapy. To address this, we evaluated the performance of the LAg-Avidity assay in a 3-assay MAA that included viral load and CD4 cell count (Table 2). In contrast to the LAg-Avidity assay alone, this MAA did not classify any samples from individuals infected >5 years as MAA positive and had a shadow <1 year (158 days). Our analysis indicated that this MAA had a mean window period of only 85 days, which is lower than the mean window period of 130 days (95% CI: 118–142) provided in the current package insert [24].

thumbnail

Table 2. Performance characteristics of MAAs that include the LAg-Avidity assay and HIV viral load, with and without CD4 cell count.

doi:10.1371/journal.pone.0082772.t002

For comparison, we evaluated the performance of the LAg-Avidity assay in 2-assay MAAs that included viral load without CD4 cell count (Table 2). For this analysis, we evaluated two MAAs that included the LAg-Avidity assay (using an assay cutoff of 1.0 or 1.5 OD-n) with viral load (assay cutoff: of 1,000 copies/mL). A viral load cutoff of 1,000 copies/mL was used previously by the developers of the LAg-Avidity assay to identify individuals with viral suppression [36]. These 2-assay MAAs classified 2–4% of the samples from individuals infected for 4–5 years and some individuals infected >5 years as MAA positive. In contrast to the LAg-Avidity assay alone (Figure 1), these two MAAs did converge to 0% MAA positive by 8 years (data not shown). These MAAs had mean window period of 80 and 134 (using LAg-avidity assay cutoffs of 1.0 and 1.5, respectively). However, the shadows for these MAAs were >1 year (446 and 690 days respectively), indicating that these MAAs were estimating incidence over more than a year prior to sample collection. Therefore, these MAAs were not evaluated further.

Comparison of the performance of optimized MAAs

The next step in the analysis was to compare the performance of >30,000 testing algorithms that included the LAg-Avidity assay alone or in combination with up to three other assays, using a range of cutoffs for each assay (see Methods). For each class of testing algorithms that met pre-set requirements (see Methods), we identified the MAA that had the largest mean window period (referred to as optimized MAAs). None of the algorithms using the LAg-Avidity assay alone or the LAg-Avidity assay with viral load met pre-set performance requirements (see Methods). The optimized 2-assay MAA included the LAg-Avidity and BioRad-Avidity assays (MAA #1, Figure 2); this MAA had a mean window period of 119 days. The optimized 4-assay MAA included the LAg-Avidity assay, the BioRad-Avidity assay, CD4 cell count, and viral load (MAA #2, Figure 2); this MAA had a mean window period of 146 days and was the top-performing MAA among >30,000 algorithms evaluated. Interestingly, our search did not identify a 3-assay MAA that was superior to either the optimized 2-assay MAA or the optimized 4-assay MAA (considering mean window period and relative cost).

thumbnail

Figure 2. Optimized multi-assay algorithms (MAAs).

The figure shows two optimized MAAs that include the LAg-Avidity assay. The assays, assay cutoffs, window periods, and shadows are shown for each MAA. The 95% confidence intervals for the window periods and shadows are shown in parentheses. Abbreviations: LAg-Avidity: limiting antigen (LAg) avidity assay; BioRad-Avidity: BioRad-Avidity assay; OD-n: normalized optical density. The following units are used for the component assays: LAg-Avidity assay: OD-n; BioRad-Avidity assay: percentage (avidity index); CD4 cell count: cells/mm3; viral load: copies/mL.

doi:10.1371/journal.pone.0082772.g002

Because the target antigens used in the LAg-Avidity assay and BED-CEIA are similar [12], [17], we did not include the BED-CEIA in our initial analyses. When we expanded the search to include the BED-CEIA, the best MAA identified (from among >500,000 MAAs evaluated) included the BED-CEIA (assay cutoff 1.6 OD-n), the BioRad-Avidity assay (assay cutoff 85%), CD4 cell count (assay cutoff: 200 cells/mm3), and viral load (assay cutoff: 400 copies/mL); this MAA is identical to MAA #2 (Figure 2), except that the LAg-Avidity assay is replaced with the BED-CEIA. Because the 4-assay MAA that includes the BED-CEIA is identical to the optimized 4-assay MAA described in a previous report [14], we did not perform any additional analyses with that MAA.

Performance characteristics of the optimized 2-assay and 4-assay MAAs (MAA #1 and MAA #2, Figure 2) are shown in Table 3. Probability curves for these MAAs are shown in Figure 1B. The proportion of samples identified as MAA positive converged to zero for both MAAs, and none of the 845 samples from individuals infected >5 years was classified as MAA positive (Table 3). Because the 2-assay MAA has a shorter mean window period than the 4-assay MAA (119 days vs. 146 days), larger cross-sectional surveys would be required to obtain incidence estimates with the same precision (relative sample size: 123% compared to MAA #2).

thumbnail

Table 3. Performance characteristics of optimized MAAs that include the LAg-Avidity assay.

doi:10.1371/journal.pone.0082772.t003

Relative cost of optimized MAAs

We compared the relative cost of sample analysis using the optimized 2-assay and 4-assay MAAs (Table 3, see Methods). Relative cost was also adjusted for the mean window period of each MAA, to account for differences in the sample sizes that would be needed to obtain incidence estimates with similar precision. This analysis assumed that testing would be performed using the assays in the order shown in Figure 2. The cost of the 2-assay MAA was less than a third of the cost of the 4-assay MAA, even after adjusting for the shorter mean window period.

Of note, one of the MAAs evaluated in the initial search of over >30,000 algorithms was the 3-assay MAA described above that included the LAg-Avidity assay (cutoff <1.5 OD-n), viral load (cutoff >1,000 copies/mL) and CD4 cell count (cutoff >200 cells/mm3, Table 2) [24]. The implication of the lower mean window period (85 vs. 130 days) is that an incidence survey would have to be 72% larger using this MAA than one using the optimized 4-assay MAA (mean window period 146 days), and 40% larger than one using the optimized 2-assay MAA (mean window period 119 days). Furthermore, the adjusted relative cost of this 3-assay MAA (using a window period of 85 days) was greater than the costs of both the optimized 2 and 4-assay MAAs, which makes it undesirable. Even if the manufacturer's recommended mean window period of 130 days were used to adjust the relative cost of the 3-assay MAA, it would still be more costly than both of the optimized MAAs (adjusted relative cost: 342% compared to the optimized 2-assay MAA, 106% compared to the optimized 4-assay MAA).

Comparison of incidence estimates obtained using optimized MAAs to those obtained from cohort follow-up

As a final step in our analysis, we used the LAg-Avidity assay alone and selected MAAs to generate cross-sectional incidence estimates for three cohorts: HPTN 064, HIVNET 001, and HPTN 061 (see Methods, Table 4). These incidence estimates were compared to longitudinal incidence based on cohort follow-up [30], [31], [37]. When the LAg-Avidity assay was used alone with a cutoff of 1.0 or 1.5 OD-n (cutoffs recommended by the manufacturer), the 95% CIs of the incidence estimates did not cover the longitudinal incidence point estimates for one of the three cohorts (HPTN 061). In contrast, the 95% CIs for all three of the MAAs evaluated covered the longitudinal incidence point estimates for all three cohorts. In addition, the incidence estimates obtained using the MAAs were much closer to the longitudinal estimates than the incidence estimates obtained using the LAg-Avidity assay alone for the two cohorts that included individuals with long-term infection (i.e., individuals who were HIV infected at enrollment, HPTN 064 and HPTN 061, % difference, Table 4). The optimized 4-assay MAA provided incidence estimates that were closest to the longitudinal incidence estimates for all three cohorts (cross-sectional vs. longitudinal estimates: HPTN 064: 0.26% vs. 0.24%; HIVNET 001: 1.09% vs. 1.04%; HPTN 061: 3.44% vs. 3.02%).

thumbnail

Table 4. HIV incidence estimates for three clinical cohorts in the United States.

doi:10.1371/journal.pone.0082772.t004

Discussion

This report includes a comprehensive evaluation of the performance of the LAg-Avidity assay alone and MAAs that include the LAg-Avidity assay. When used in a single-assay format, the LAg-Avidity assay classified a smaller proportion of samples from individuals with long-term infection as assay positive than the BED-CEIA alone (based on comparison to results previously obtained for the BED-CEIA using the same sample sets [14]). However, the LAg-Avidity assay still classified some individuals with long-term infection as assay positive and failed to provide accurate HIV incidence estimates in cohorts that included individuals with long-term infections (HPTN 064 and HPTN 061). Some individuals who were infected >5 years were also classified as MAA positive when the LAg-Avidity assay was used in 2-assay MAAs that also included viral load. Furthermore, MAAs that included only the LAg-Avidity assay and viral load had shadows that were >1 year, indicating that they were estimating incidence more than a year prior to sample collection.

Using an objective, statistical approach to evaluate >30,000 different testing algorithms that included the LAg-Avidity assay, we identified an optimized 2-assay MAA and an optimized 4-assay MAA. Both of these MAAs include the LAg-Avidity assay and the BioRad-Avidity assay; the 4-assay MAA also includes CD4 cell count and viral load. Both of the optimized MAAs classified all 845 samples from individuals infected >5 years as MAA negative. Furthermore, both of the optimized MAAs provided cross-sectional incidence estimates that were similar to those obtained by longitudinal follow-up for three cohorts in the US with varying HIV incidence. The 4-assay MAA was the top-performing MAA among >30,000 MAAs that did not include the BED-CEIA; this MAA provided incidence estimates that were nearly identical to those obtained by cohort follow-up. Of note, the performance of the optimized 4-assay MAA described in this report was very similar to that of an optimized 4-assay MAA which was the best MAA in the expanded search of >500,000 algorithms; that MAA included the BED-CEIA rather than the LAg-Avidity assay (described previously [14], [16]). These two MAAs use the same cutoffs for the BioRad-Avidity assay, CD4 cell count, and viral load.

Acceptable incidence estimates were also obtained using a 3-assay MAA that included the LAg-Avidity assay, CD4 cell count, and viral load. The cutoffs used for the LAg-Avidity assay (1.5 OD-n) and CD4 cell count (200 cells/mm3) in this MAA are recommended by the manufacturer [24], the cutoff used for viral load (1,000 copies/mL) was used by the developers of the LAg-Avidity assay in a previous study (in conjunction with self-reported use of antiretroviral therapy (ART) [36]. We did not include self-report of ART in our algorithms; recent reports indicate that self-report of ART use is often unreliable [38][40]. This 3-assay MAA did not classify any of samples from individuals infected >5 years as MAA positive and had a shadow <1 year. However, our analysis indicated that this MAA had a mean window period of only 85 days, which is shorter than the mean window period of 130 days (95% CI: 118–142) provided in the current package insert [24] and is shorter than the mean window periods of the optimized 2-assay and 4-assay MAAs (119 and 146 days, respectively). The implication of the lower mean window period is that a cross sectional survey to measure incidence would have to be larger using the 3-assay MAA than one using either the optimized 2-assay or the optimized 4-assay MAA. Furthermore, the adjusted relative cost for sample testing using this 3-assay MAA was greater than the cost for sample testing using the optimized 4-assay MAA, which makes it difficult to justify its use.

We recognize that the confidence intervals for incidence obtained using the MAAs (which incorporate uncertainty in the mean window period) are larger than those obtained by longitudinal follow-up of cohorts. Additional studies that improve the precision of the mean window period would reduce the width of these confidence intervals. The width of the confidence intervals could also be reduced with larger cross-sectional surveys. Furthermore, even with larger cross-sectional surveys, it may be less expensive to estimate incidence using a MAA than a cohort study because no longitudinal follow-up is required.

Our identification of an optimized 2-assay MAA that does not require CD4 cell count or HIV viral load has major implications for global HIV surveillance. The optimized 2-assay MAA identified in this report, which includes the LAg-Avidity and BioRad-Avidity assays without CD4 or viral load, has lower testing costs than the optimized 4-assay MAA, even after accounting for the larger sample sizes that would be needed with the 2-assay MAA because of its shorter mean window period. Note that this cost comparison only included costs for laboratory testing; costs associated with obtaining samples for analysis were not included in this analysis. Because the 2-assay MAA does not require CD4 cell count data, it can be performed entirely using stored plasma or serum samples, which would be an advantage in some studies. We previously described an optimized 3-assay MAA that included the BED-CEIA, the BioRad-Avidity assay, and viral load (without CD4 cell count) [14]; an advantage of the 2-assay MAA described in this report over the 3-assay MAA described in the previous report [14] is that the 2-assay MAA also does not require viral load testing. Furthermore, recent data suggests that both the LAg-Avidity and BioRad-Avidity assays can be performed using dried blood spots [18], [41]. This could significantly reduce the complexity and cost of sample acquisition, shipping and storage in incidence surveys.

The samples used in this report were obtained from diverse cohorts that included men and women of different ethnicities and ages; and individuals infected through heterosexual, homosexual and parenteral routes. All of these samples were from the US where the most prevalent HIV-1 subtype is B. Serologic assays developed for cross-sectional incidence estimation have been shown to perform differently depending on the infecting subtype [7], [42], [43]. Therefore, additional studies are needed to evaluate the performance of the LAg-Avidity assay (alone and in MAAs) in populations where other HIV-1 subtypes are prevalent.

Acknowledgments

The authors thank the study teams and participants of the cohort studies that provided samples and data for this work. The authors also thank Caroline Mullis, Amy Oliver, Kevin Eaton, Jacquie Astemborski, and Xiuhong Li for technical support, and thank Deborah Donnell for helpful discussions. Disclaimer, the findings and conclusions in this article are those of the authors and do not necessarily represent the views of the National Institutes of Health. Use of trade names is for identification purposes only and does not constitute endorsement by the National Institutes of Health and Prevention or the Department of Health and Human Services. The views expressed here do not necessarily reflect the official policies of the City and County of San Francisco, nor does mention of the San Francisco Dept. of Public Health imply its endorsement.

Author Contributions

Conceived and designed the experiments: JK RB SHE OL. Performed the experiments: AL OL. Analyzed the data: JK RB MMC . Contributed reagents/materials/analysis tools: CC SPB GRS GDK RDM SHM JBM JB KHM BAK JEJ SLH TCQ. Wrote the paper: JK RB AFL MMC CC SPB GRS GDK RDM SHM JBM JB KHM BAK JEJ SLH TCQ SHE OL.

References

  1. 1. Brookmeyer R (2010) Measuring the HIV/AIDS epidemic: approaches and challenges. Epidemiol Rev 32: 26–37. doi: 10.1093/epirev/mxq002
  2. 2. Mastro TD, Kim AA, Hallett T, Rehle T, Welte A, et al. (2010) Estimating HIV Incidence in Populations Using Tests for Recent Infection: Issues, Challenges and the Way Forward. J HIV AIDS Surveill Epidemiol 2: 1–14.
  3. 3. Busch MP, Pilcher CD, Mastro TD, Kaldor J, Vercauteren G, et al. (2010) Beyond detuning: 10 years of progress and new challenges in the development and application of assays for HIV incidence estimation. AIDS 24: 2763–2771. doi: 10.1097/qad.0b013e32833f1142
  4. 4. Brookmeyer R, Quinn T, Shepherd M, Mehendale S, Rodrigues J, et al. (1995) The AIDS epidemic in India: a new method for estimating current human immunodeficiency virus (HIV) incidence rates. Am J Epidemiol 142: 709–713.
  5. 5. Laeyendecker O, Brookmeyer R, Mullis CE, Donnell D, Lingappa J, et al. (2012) Specificity of four laboratory approaches for cross-sectional HIV incidence determination: analysis of samples from adults with known nonrecent HIV infection from five African countries. AIDS Res Hum Retroviruses 28: 1177–1183. doi: 10.1089/aid.2011.0341
  6. 6. Laeyendecker O, Brookmeyer R, Oliver AE, Mullis CE, Eaton KP, et al. (2012) Factors associated with incorrect identification of recent HIV infection using the BED capture immunoassay. AIDS Res Hum Retroviruses 28: 816–822. doi: 10.1089/aid.2011.0258
  7. 7. Mullis CE, Munshaw S, Grabowski MK, Eshleman SH, Serwadda D, et al. (2013) Differential specificity of HIV incidence assays in HIV subtypes A and D-infected individuals from Rakai, Uganda. AIDS Res Hum Retroviruses. Epub available online. doi: 10.1089/aid.2012.0105
  8. 8. Marinda ET, Hargrove J, Preiser W, Slabbert H, van Zyl G, et al. (2010) Significantly diminished long-term specificity of the BED capture enzyme immunoassay among patients with HIV-1 with very low CD4 counts and those on antiretroviral therapy. J Acquir Immune Defic Syndr 53: 496–499. doi: 10.1097/qai.0b013e3181b61938
  9. 9. Hayashida T, Gatanaga H, Tanuma J, Oka S (2008) Effects of low HIV type 1 load and antiretroviral treatment on IgG-capture BED-enzyme immunoassay. AIDS Res Hum Retroviruses 24: 495–498. doi: 10.1089/aid.2007.0150
  10. 10. Murphy G, Parry JV (2008) Assays for the detection of recent infections with human immunodeficiency virus type 1. Euro Surveill 13: pii18966.
  11. 11. Guy R, Gold J, Calleja JM, Kim AA, Parekh B, et al. (2009) Accuracy of serological assays for detection of recent infection with HIV and estimation of population incidence: a systematic review. Lancet Infect Dis 9: 747–759. doi: 10.1016/s1473-3099(09)70300-7
  12. 12. Dobbs T, Kennedy S, Pau CP, McDougal JS, Parekh BS (2004) Performance characteristics of the immunoglobulin G-capture BED-enzyme immunoassay, an assay to detect recent human immunodeficiency virus type 1 seroconversion. J Clin Microbiol 42: 2623–2628. doi: 10.1128/jcm.42.6.2623-2628.2004
  13. 13. Masciotra S, Dobbs T, Candal D, Hanson D, Delaney K, et al.. (2010) Antibody avidity-based assay for identifying recent HIV-1 infection based on a genetic systems 1/2 + O EIA. 17th Conference on Retroviruses and Opportunistic Infections. February 16–19. San Francisco, CA: Abstract 937.
  14. 14. Brookmeyer R, Konikoff J, Laeyendecker O, Eshleman SH (2013) Estimation of HIV incidence using multiple biomarkers. Am J Epidemiol 177: 264–272. doi: 10.1093/aje/kws436
  15. 15. Cousins MM, Laeyendecker O, Beauchamp G, Donnell D, Coates T, et al.. (2012) Multi-region analysis may enhance the utility of HIV diversity as a biomarker for HIV incidence determination. 19th Conference on retroviruses and Opportunistic Infections. March 5–8. Seattle, WA: Abstract 539.
  16. 16. Brookmeyer R, Laeyendecker O, Donnell D, Eshleman SE (2013) Cross-Sectional HIV incidence estimation in HIV prevention research. J Acquir Immune Defic Syndr 63: S233–S239. doi: 10.1097/qai.0b013e3182986fdf
  17. 17. Wei X, Liu X, Dobbs T, Kuehl D, Nkengasong JN, et al. (2010) Development of two avidity-based assays to detect recent HIV type 1 seroconversion using a multisubtype gp41 recombinant protein. AIDS Res Hum Retroviruses 26: 61–71. doi: 10.1089/aid.2009.0133
  18. 18. Duong YT, Qiu M, De AK, Jackson K, Dobbs T, et al. (2012) Detection of recent HIV-1 infection using a new limiting-antigen avidity assay: potential for HIV-1 incidence estimates and avidity maturation studies. PLoS One 7: e33328. doi: 10.1371/journal.pone.0033328
  19. 19. Centers for Disease Control and Prevention (2010) Prevalence and awareness of HIV infection among men who have sex with men — 21 cities, United States, 2008. MMWR Morb Mortal Wkly Rep 59: 1201–1207.
  20. 20. Freiden T (2013) Remarks from the Director of the US Centers for Disease Control and Prevention. 20th Conference on Retroviruses and Oppurtunistic Infections. Atlanta, GA: CROI, LLC.
  21. 21. Kassanjee R, Murphy G, Busch M, Pilcher CD, McKinney E, et al.. (2013) The performance of candidate assays to detect HIV infection for cross-sectional incidence estimation: an independent , comparative evaluation. 20th Conference on Retroviruses and Opportunistic Infections March 3–6. Atlanta, GA: Abstract 1056.
  22. 22. Mastro TD (2013) Determining HIV incidence in populations: Moving in the right direction. J Infect Dis 207: 204–206. doi: 10.1093/infdis/jis661
  23. 23. Sedia Biosciences Corporation (2013) Sedia HIV-1 LAg-Avidity EIA. LN 6039, 04 ed. Portland, OR.
  24. 24. Sedia Biosciences Corporation (2013) Sedia HIV-1 LAg-Avidity EIA. LN 6039, 05 ed. Portland, OR.
  25. 25. Kaslow RA, Ostrow DG, Detels R, Phair JP, Polk BF, et al. (1987) The Multicenter AIDS Cohort Study: rationale, organization, and selected characteristics of the participants. Am J Epidemiol 126: 310–318. doi: 10.1093/aje/126.2.310
  26. 26. Vlahov D, Anthony JC, Munoz A, Margolick J, Nelson KE, et al. (1991) The ALIVE study, a longitudinal study of HIV-1 infection in intravenous drug users: description of methods and characteristics of participants. NIDA Res Monogr 109: 75–100.
  27. 27. Celum CL, Buchbinder SP, Donnell D, Douglas JM Jr, Mayer K, et al. (2001) Early human immunodeficiency virus (HIV) infection in the HIV Network for Prevention Trials Vaccine Preparedness Cohort: risk behaviors, symptoms, and early plasma and genital tract virus load. J Infect Dis 183: 23–35. doi: 10.1086/317658
  28. 28. Moore RD (1998) Understanding the clinical and economic outcomes of HIV therapy: the Johns Hopkins HIV clinical practice cohort. J Acquir Immune Defic Syndr Hum Retrovirol 17(Suppl 1): S38–41. doi: 10.1097/00042560-199801001-00011
  29. 29. Laeyendecker O, Brookmeyer R, Cousins MM, Mullis CE, Konikoff J, et al. (2013) HIV incidence determination in the United States: a multiassay approach. J Infect Dis 207: 232–239. doi: 10.1093/infdis/jis659
  30. 30. Koblin B, Mayer K, Eshleman SH, Wang L, Shoptaw S, et al. (2013) Correlates of HIV acquisition in a cohort of Black men who have sex with men in the United States: HIV Prevention Trials Network (HPTN) 061. PLoS One In Press. doi: 10.1371/journal.pone.0070413
  31. 31. Hodder SL, Justman J, Hughes JP, Wang J, Haley DF, et al. (2013) HIV acquisition among women from selected areas of the United States: a cohort study. Ann Intern Med 158: 10–18. doi: 10.7326/0003-4819-158-1-201301010-00004
  32. 32. Fiebig EW, Wright DJ, Rawal BD, Garrett PE, Schumacher RT, et al. (2003) Dynamics of HIV viremia and antibody seroconversion in plasma donors: implications for diagnosis and staging of primary HIV infection. AIDS 17: 1871–1879. doi: 10.1097/00002030-200309050-00005
  33. 33. Kaplan EH, Brookmeyer R (1999) Snapshot estimators of recent HIV incidence rates. Oper Res 47: 29–37. doi: 10.1287/opre.47.1.29
  34. 34. Brookmeyer R (2010) On the statistical accuracy of biomarker assays for HIV incidence. J Acquir Immune Defic Syndr 54: 406–414. doi: 10.1097/qai.0b013e3181dc6d2c
  35. 35. Brookmeyer R (1997) Accounting for follow-up bias in estimation of human immunodeficiency virus incidence rates. J Roy Stat Soc a Sta 160: 127–140. doi: 10.1111/1467-985x.00049
  36. 36. Parekh B, Duong Y, Mavengere Y, Dobbs T, Chang J, et al.. (2012) Performance of new LAg-Avidity EIA to measure HIV-1 incidence in a cross-sectional population: Swaziland HIV incidence measurement survey (SHIMS). XIX International AIDS Conference. July 22–27. Washington DC: Abstract LBPE27.
  37. 37. Celum C, Buchbinder S, Donnell D, Douglas J, Mayer K, et al. (2001) Early human immunodeficiency virus (HIV) infection in the HIV network for prevention trials vaccine preparedness cohort: Risk behaviors, symptoms, and early plasma and genital tract virus load. J Infect Dis 183: 23–35. doi: 10.1086/317658
  38. 38. Sullivan AK, Savage EJ, Lowndes CM, Paul G, Murphy G, et al. (2013) Non-disclosure of HIV status in UK sexual health clinics–a pilot study to identify non-disclosure within a national unlinked anonymous seroprevalence survey. Sex Transm Infect 89: 120–121. doi: 10.1136/sextrans-2012-050801
  39. 39. Fogel JM, Wang L, Parsons TL, Ou S-S, Piwowar-Manning E, et al. (2013) Undisclosed antiretroviral drug use in a multi-national clinical trial (HPTN 052). J Infect Dis In press.
  40. 40. Kahle EM, Kashuba A, Baeten JM, Fife K, Celum C, et al. (2013) Unreported antiretroviral use by HIV-1 infected participants enrolling in a prospective research study. J Acquir Immune Defic Syndr In press. doi: 10.1097/qai.0b013e3182a2db02
  41. 41. Masciotra S, Wei X, Luo W, Smith A, Miles I, et al.. (2013) Performance of dried blood spots with a modified Genetic Systems HIV 1/2 +plus O avidity based assay for idenitfying recent HIV infections. 20th Conference on Retroviruses and Opportunistic Infections. March 3–6. Atlanta, GA: Abstract 1055.
  42. 42. Young CL, Hu DJ, Byers R, Vanichseni S, Young NL, et al. (2003) Evaluation of a sensitive/less sensitive testing algorithm using the bioMerieux Vironostika-LS assay for detecting recent HIV-1 subtype B′ or E infection in Thailand. AIDS Res Hum Retroviruses 19: 481–486. doi: 10.1089/088922203766774522
  43. 43. Parekh BS, Hanson DL, Hargrove J, Branson B, Green T, et al. (2011) Determination of mean recency period for estimation of HIV type 1 Incidence with the BED-capture EIA in persons infected with diverse subtypes. AIDS Res Hum Retroviruses 27: 265–273. doi: 10.1089/aid.2010.0159