Conceived and designed the experiments: AG GK ALR AB EH. Performed the experiments: AG EV SS RL GK AB. Analyzed the data: AG. Contributed reagents/materials/analysis tools: AG EH. Wrote the paper: AG EH.
None of the authors has any relationships or support that might be perceived as constituting a competing interest, except that Quidel Corporation covered the costs of airfare and hotel for AG at one conference in February 2009.
Influenza is major public health threat worldwide, yet the diagnostic accuracy of rapid tests in developing country settings is not well described.
To investigate the diagnostic accuracy of the QuickVue Influenza A+B test in a primary care setting in a developing country, we performed a prospective study of diagnostic accuracy of the QuickVue Influenza A+B test in comparison to reverse transcriptase-polymerase chain reaction (RT-PCR) in a primary healthcare setting in children aged 2 to 12 years in Managua, Nicaragua. The sensitivity and specificity of the QuickVue test compared to RT-PCR were 68.5% (95% CI 63.4, 73.3) and 98.1% (95% CI 96.9, 98.9), respectively, for children with a fever or history of a fever and cough and/or sore throat. Test performance was found to be lower on the first day that symptoms developed in comparison to test performance on days two or three of illness.
Our study found that the QuickVue Influenza A+B test performed as well in a developing country primary healthcare facility setting as in developed country settings.
Influenza is a major health threat throughout the world
Logistical constraints in developing countries largely dictate the types of laboratory tests available for use in influenza surveillance. While reverse transcriptase-polymerase chain reaction (RT-PCR) and viral isolation are the preferred diagnostic methods, rapid influenza tests are valuable for their ease of use and laboratory-independence. In the developed world, rapid influenza tests have been evaluated in both laboratory and clinical settings, yielding sensitivities of 45–90% and specificities of 86–100%
Historically, viral isolation has been regarded as the “gold standard” for diagnosis of influenza infection. However, recently a number of studies have demonstrated the superior sensitivity of RT-PCR for the detection of influenza viruses
To examine the diagnostic accuracy of the QuickVue Influenza A+B test in comparison to RT-PCR in children treated in a primary healthcare facility in a developing country, we performed a prospective study in Managua, Nicaragua and report the results here.
This study was approved by the institutional review boards at the University of California (UC) Berkeley and the Nicaraguan Ministry of Health. Written informed consent was obtained from the parents or guardians of all participants.
This study was performed as part of the Nicaraguan Influenza Cohort Study (NICS), a prospective community-based cohort study of approximately 3,800 children 2–14 years old in Managua, Nicaragua
All specimens collected between January 1, 2008 and December 31, 2008 from children in the NICS who presented with a fever, or history of fever or feverishness, and cough and/or sore throat within five days of symptom onset were included in the study. Information from all medical appointments was systematically collected using standardized data collection forms. Data included age, sex, two measurements of temperature, date of fever onset, and other symptoms. Upon presentation with testing criteria, children were randomly selected for respiratory testing using a random number generator and then referred to the HCSFV clinical laboratory. There, a nasal swab was first collected for the rapid test, QuickVue Influenza A+B, using the swab provided in the kit. Then nasal and throat Dacron swabs were collected and placed into one tube containing three mL of viral transport medium for RT-PCR and viral culture. The QuickVue test was performed onsite immediately. Samples for RT-PCR and viral isolation collected on workdays (Monday–Saturday) were kept at 4°C and processed within 16 hours. Samples collected on Sundays and holidays were stored for up to 48 hours prior to processing.
QuickVue Influenza A+B is a lateral-flow immunoassay that detects and differentiates between influenza A and B. Tests were performed according to the manufacturer's instructions. Tests were read after exactly ten minutes and were considered negative if, at ten minutes, a pink line had not appeared to indicate a positive influenza A or B result. Tests were considered invalid if the blue control line did not appear or if the background color made it impossible to read the test result. Invalid tests were repeated once.
Samples were transported to the National Virology Laboratory at the Nicaraguan Ministry of Health, where RNA was extracted from nasal swabs using the QIAamp® Viral RNA Mini Kit. (Qiagen Sciences, Germantown, MD) Influenza viruses A and B were amplified using Promega Access RT-PCR (Promega Corporation, Madison, WI) and typed in-house according to a protocol provided by the Centers for Disease Control and Prevention (CDC). Laboratory technicians were trained in how to perform RT-PCR, including attendance at influenza testing workshops held by the CDC and World Health Organization and hands-on training by study personnel from UC Berkeley. Laboratory technicians performing RT-PCR testing were blinded to the results of the rapid test and to the patients' clinical presentation.
To replicate the typical training resources that would be available if rapid tests were widely distributed in a developing country setting, a 1.5-hour training session was held. This training consisted of a short presentation explaining the symptoms and burden of influenza, influenza surveillance, collection of nasal swabs, principles of the QuickVue rapid test, and instructions on how to perform the test. Laboratory technicians then practiced taking one nasal swab specimen and performing a rapid test. Prior to the training session, technicians had never performed an influenza rapid test and many had little or no experience collecting nasal swab specimens.
The main outcomes of this study were the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of the QuickVue Influenza A+B test compared with RT-PCR. Agreement between tests was calculated using the kappa statistic. Sub-analyses were performed to examine test performance stratified by testing criteria, day of presentation, age, and sex. All statistical analyses were performed using STATA 10.1 (StataCorp, College Station, Texas).
Between January 1, 2008, and December 31, 2008, 3,916 children participated in the cohort study. During this period, study participants attended 13,666 medical visits, of which 3,935 (29%) met the testing criteria. Of these, 1,195 (30%) were randomly selected for influenza testing, and 1,157 duplicate specimens (97%) were collected from 989 children and tested by both rapid test and RT-PCR, while 38 children received a rapid test only (
Flowchart diagramming the selection of samples used to analyze the performance of the QuickVue Influenza A+B rapid test. Due to complications related to transport and sample separation, samples (n = 37) collected during holiday periods lasting longer than 3 days did not undergo RT-PCR and therefore were not included in this study.
All cases n = 1,157 | Cases with influenza-like illness n = 578 | |
Sex | ||
Female | 581 (50.2) | 301 (52.1) |
Age (years) | ||
2 | 84 (7.3) | 42 (7.3) |
3 | 169 (14.6) | 92 (15.9) |
4 | 166 (14.4) | 79 (13.7) |
5 | 158 (13.7) | 76 (13.2) |
6 | 129 (11.2) | 75 (13.0) |
7 | 113 (9.8) | 63 (10.9) |
8 | 113 (9.8) | 52 (9.0) |
9 | 79 (6.8) | 34 (5.9) |
10 | 53 (4.6) | 23 (4.0) |
11 | 53 (4.6) | 26 (4.5) |
12 | 36 (3.1) | 15 (2.6) |
13 | 4 (0.4) | 1 (0.2) |
Data on the agreement of RT-PCR and rapid test are presented in
Patients with fever or a history of fever and cough and/or sore throat (n = 1,157) | |||||
RT-PCR | Agreement (%) | Kappa (95% CI) | |||
+ | − | ||||
QuickVue | + | 246 | 15 | 88.9 | 0.72 (0.67, 0.76) |
− | 113 | 783 | |||
Patients with a temperature ≥37.8°C and cough and/or sore throat (n = 578) | |||||
RT-PCR | Agreement (%) | Kappa (95% CI) | |||
+ | − | ||||
QuickVue | + | 189 | 6 | 86.3 | 0.72 (0.65, 0.77) |
− | 73 | 310 |
Number Positive | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value | |||||
(%) | (95% CI) | (%) | (95% CI) | (%) | (95% CI) | (%) | (95% CI) | ||
Children presenting with a fever or reported fever and cough and/or sore throat (n = 1,157) | |||||||||
Influenza | 359 | 68.5 | (63.4, 73.3) | 98.1 | (96.9, 98.9) | 94.3 | (90.7, 96.7) | 87.4 | (85.0, 89.5) |
Influenza A | 224 | 65.2 | (58.5, 71.4) | 99.1 | (98.3, 99.6) | 94.8 | (90.0, 97.7) | 92.2 | (90.4, 93.8) |
Influenza B | 135 | 74.1 | (65.8, 81.2) | 99.1 | (98.3, 99.6) | 91.7 | (84.9, 96.2) | 96.7 | (95.4, 97.7) |
Children presenting with a fever ≥37.8°C and cough and/or sore throat (n = 578) | |||||||||
Influenza | 262 | 72.1 | (66.3, 77.5) | 98.1 | (95.9, 99.3) | 96.9 | (93.4, 98.9) | 80.9 | (76.6, 84.8) |
Influenza A | 169 | 69.8 | (62.3, 76.6) | 99.3 | (97.9, 99.8) | 97.5 | (92.9, 99.5) | 88.8 | (85.6, 91.6) |
Influenza B | 93 | 76.3 | (66.4, 84.5) | 99.2 | (97.9, 99.8) | 94.7 | (86.9, 98.5) | 95.6 | (93.5, 97.2) |
Limiting the analysis to the 578 samples from patients who met the CDC definition of influenza-like illness, defined as a fever of ≥37.8°C plus cough or sore throat, resulted in a slightly higher sensitivity for the rapid test but did not significantly change the overall test performance (
Sample Number | Number Positive | Sensitivity | Specificity | Positive Predictive Value | Negative Predictive Value | |||||
(%) | (95% CI) | (%) | (95% CI) | (%) | (95% CI) | (%) | (95% CI) | |||
Patients with fever or a history of fever and cough and/or sore throat (n = 1,157) | ||||||||||
Day 1 | 263 | 79 | 51.9 | (40.4, 63.3) | 98.4 | (95.3, 99.7) | 93.2 | (81.3, 98.6) | 82.6 | (77.0, 87.4) |
Day 2 | 578 | 189 | 75.1 | (68.3, 81.1) | 97.9 | (96.0, 99.1) | 94.7 | (89.8, 97.7) | 89 | (85.7, 91.8) |
Day 3 | 211 | 66 | 74.2 | (62.0, 84.2) | 97.9 | (94.1, 99.6) | 94.2 | (84.1, 98.8) | 89.3 | (83.4, 93.6) |
Day 4 | 90 | 19 | 57.9 | (33.5, 79.7) | 98.6 | (92.4, 100) | 91.7 | (61.5, 99.8) | 89.7 | (80.8, 95.5) |
Patients with a fever ≥37.8°C and cough and/or sore throat (n = 578) | ||||||||||
Day 1 | 182 | 62 | 59.7 | (46.4, 71.9) | 97.5 | (92.9, 99.5) | 92.5 | (79.6, 98.4) | 82.4 | (75.1, 88.3) |
Day 2 | 284 | 147 | 76.9 | (69.2, 83.4) | 98.5 | (94.8, 99.8) | 98.3 | (93.9, 99.8) | 79.9 | (73.0, 85.6) |
Day 3 | 78 | 39 | 79.5 | (63.5, 90.7) | 97.4 | (86.5, 99.9) | 96.9 | (83.8, 99.9) | 82.6 | (68.6, 92.2) |
Day 4 | 26 | 11 | 54.5 | (23.4, 83.3) | 100 | (78.2, 100) | 100 | (54.1, 100) | 75.0 | (50.9, 91.3) |
Note: Day 1 refers to the day of symptom onset.
Our study found that the QuickVue Influenza A+B test performed with a moderate sensitivity (68.5%) and high specificity (98.1%) in comparison to RT-PCR in a developing country primary care setting. Laboratory technicians found the test easy to use and needed minimal training to adequately perform the test.
In the US and other developed countries, the QuickVue Influenza A+B rapid test has been found to have a sensitivity of 67%–85% and specificity of 97%–100%
Importantly, a substantial number of participants (23%) presented on the first day of illness, and the sensitivity of the rapid test was found to be lower among children who presented and were tested on the first day. Few studies of rapid tests have captured data on these early cases. In a study in Hong Kong, researchers examined the performance of the QuickVue rapid test and did not find a significant difference in the first 24 hours following symptom onset
The large number of children presenting so early resulted from the cohort study protocol, in which families of participants were asked to bring in their children at the first sign of illness. However, persons with influenza typically do not present until several days into the illness. Therefore, it is likely that the sensitivity and resulting positive predictive values we observed are underestimates of the test's performance in the clinical setting, since it is less common for patients to present on the day of symptom onset. However, during an influenza pandemic, patients may present very early following symptom onset; thus, it is imperative to evaluate the performance of any rapid test in detecting influenza very early after onset of illness.
One of the limitations of this study is the age range of the patients. Because the study was restricted to children aged 2–14 years, the sensitivities and specificities we found apply only to this group. However, because the rapid test performed at approximately the same level in this study as in studies performed in children in developed country settings, the rapid test may well perform at the same level across all age ranges in developing country settings as observed in studies conducted in developed countries. Another limitation of this study is the use of RT-PCR as the gold standard. However, considering the very high (∼98%) specificity of the rapid test in this study and the fact that RT-PCR is substantially more sensitive than viral isolation, it is unlikely that use of RT-PCR alone, instead of a combination of RT-PCR and viral isolation, significantly affected the results.
In conclusion, the rapid influenza test we used performed well in a primary healthcare setting in Managua, Nicaragua. Clinical laboratory staff needed minimal training to correctly perform the test and found the test easy to perform. Further evaluation of rapid tests in developing countries is needed to determine the feasibility of using these tests to augment existing surveillance systems and for quick deployment in the event of an influenza pandemic, as well as for use in clinical decision-making.
The authors would like to thank the Nicaraguan Ministry of Health and all of the study staff for their excellent work. In particular, we thank Maria Celina Perez Zúnigas for assistance in performing RT-PCR; Carolina Flores Hernandez, Heyri Roa Solís, Moises Navarro Martinez, and Patricia Castillo Palma for performing the influenza rapid tests, and Drs. Sergio Benito Ojeda Munguía, Luis Nery Sanchez Palma, Grethel María Navas Pasquier, and Jackeline Herrera Morales for providing medical attention to the study participants.