Surveillance for influenza and influenza-like illness (ILI) is important for guiding public health prevention programs to mitigate the morbidity and mortality caused by influenza, including pandemic influenza. Nontraditional sources of data for influenza and ILI surveillance are of interest to public health authorities if their validity can be established.
National telephone triage call data were collected through automated means for purposes of syndromic surveillance. For the 17 states with at least 500,000 inhabitants eligible to use the telephone triage services, call volume for respiratory syndrome was compared to CDC weekly number of influenza isolates and percentage of visits to sentinel providers for ILI. The degree to which the call data were correlated with either CDC viral isolates or sentinel provider percentage ILI data was highly variable among states.
Telephone triage data in the U.S. are patchy in coverage and therefore not a reliable source of ILI surveillance data on a national scale. However, in states displaying a higher correlation between the call data and the CDC data, call data may be useful as an adjunct to state-level surveillance data, for example at times when sentinel surveillance is not in operation or in areas where sentinel provider coverage is considered insufficient. Sufficient population coverage, a specific ILI syndrome definition, and the use of a threshold of percentage of calls that are for ILI would likely improve the utility of such data for ILI surveillance purposes.
Citation: Yih WK, Teates KS, Abrams A, Kleinman K, Kulldorff M, et al. (2009) Telephone Triage Service Data for Detection of Influenza-Like Illness. PLoS ONE 4(4): e5260. doi:10.1371/journal.pone.0005260
Editor: Joel Mark Montgomery, U.S. Naval Medical Research Center Detachment/Centers for Disease Control, United States of America
Received: December 22, 2008; Accepted: March 13, 2009; Published: April 17, 2009
This is an open-access article distributed under the terms of the Creative Commons Public Domain declaration which stipulates that, once placed in the public domain, this work may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose.
Funding: Centers for Disease Control and Prevention (CDC), grant numbers UR8/CCU115079 and R501PH000032. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. The authors' conclusions do not necessarily represent the views of the state health departments that contributed data.
Competing interests: The authors have declared that no competing interests exist.
The principal objective of surveillance for influenza and influenza-like illness (ILI) is to guide public health prevention programs to mitigate the morbidity and mortality caused by annual influenza epidemics, which cause approximately 36,000 deaths each year in the United States . It is hoped that these surveillance systems will also help warn of and track the development of the anticipated next influenza pandemic, which may cause over a million deaths in this country . Influenza surveillance components coordinated nationally by the CDC include: the number and percentage of all outpatient visits to sentinel providers that are for ILI, virologic surveillance for influenza virus type and subtype, mortality data reported through the 122 Cities Mortality Reporting System, pediatric hospitalization rate estimates, nationally notifiable influenza-associated pediatric deaths, and state influenza activity as reported by state and territorial epidemiologists .
Less traditional electronic data have been proposed as additional sources of surveillance information, and several authors have shown that ambulatory care and emergency department (ED) data have been useful for identifying ILI activity –. In the United Kingdom and Canada, there have been some assessments of the utility of national or provincial telephone health advice lines as a source of surveillance data for influenza and other communicable diseases –. Using weekly numbers of laboratory reports of the main respiratory pathogens, Cooper et al.  created models capable of providing weekly estimates of the proportions of calls due to specific microbiological causes, including influenza. In the U.S. as well, data from national nurse telephone triage services, although lacking laboratory confirmation, might be useful for ILI surveillance for reasons of timeliness and the potential to complement existing surveillance—it is possible that these data can be collected more quickly or efficiently than the weekly reports currently received by CDC, and they may be available in locations or at times during the year when conventional ILI surveillance systems do not operate.
For this analysis we retrospectively compared respiratory illness surveillance using electronically collected data from a national nurse telephone triage service to CDC's national influenza and ILI surveillance data for the 2004–2005 season. (For a variety of reasons, telephone triage service data from subsequent seasons were not available for this analysis.) The 2004–2005 season was characterized as “moderate” by CDC; it was associated principally with influenza A (H3N2) and peaked in February . Our goals were 1) to determine the validity of this non-traditional data source in describing the influenza season, utilizing the CDC data on number of influenza isolates and percentage of sentinel provider ILI visits as the gold standard, and 2) to consider what, if any, advantage could be gained by adding such a data source to existing national ILI surveillance indicators.
Observation period and geographic areas
We examined the period from October 3, 2004 through April 16, 2005 (week 40 of 2004 through week 15 of 2005). We compared CDC data and respiratory illness call data for the 17 states with at least 500,000 residents eligible (by virtue of their health insurance plan) to use the nurse telephone triage service. All but one of CDC's nine influenza surveillance regions were represented—three states were in the Mid-Atlantic region, three in East North Central, two in West North Central, three in South Atlantic, one in East South Central, two in West South Central, two in Mountain, and one in Pacific. The surveillance region with no states with 500,000 residents eligible to use the telephone triage service was New England.
CDC virologic and sentinel provider percentage ILI data
Two CDC influenza surveillance components were used for this comparison: the number of virologic specimens positive for influenza and the percentage of all sentinel provider visits reported as being for ILI. The absolute number of positive specimens is reported weekly by the states and includes specimens from sentinel providers as well as other sources.
The total number of patients seen for any reason and the number of those patients with ILI, from which is calculated the percentage of patients seen for ILI, are reported on a weekly basis by sentinel providers. ILI is defined as temperature of ≥100°F plus a cough and/or sore throat in the absence of a known cause other than influenza. Nationwide, 2,252 sentinel providers were enrolled in the U.S. Sentinel Provider Network during the 2004–2005 influenza season. Of these, 1,323 reported regularly, having submitted reports for at least 16 (about half) of the weeks during the reporting period of October through May. There were 1,291 sentinel providers enrolled in the 17 states in this study, of which 753 reported regularly. Having at least one regularly reporting sentinel provider per 250,000 population is CDC's goal for state ILI surveillance.
The absolute number of positive specimens from sentinel providers and other sources correlates well with CDC percentage ILI data nationally and regionally (CDC, Influenza Branch, unpublished data).
The CDC viral isolate and percentage ILI data used in this analysis were as of May 11, 2005. The numbers of sentinel providers in 2004–2005 were as of December 1, 2006.
Telephone triage service data
Optum, a national managed-care nurse telephone triage service with 26 million patients in 50 states eligible to use its services through their managed care plans, was one of several providers of aggregate data on patient diagnoses to the National Bioterrorism Syndromic Surveillance Demonstration Program (NDP) , . As part of ongoing syndromic surveillance conducted by the NDP, Optum routinely sent to the NDP's data center next-day daily counts of respiratory illness (as well as other syndromes) for each zip code with at least one caller conforming to the syndrome definition. Calls made to Optum for health information rather than for advice about a current case of illness were coded differently and did not enter the syndromic surveillance system.
The flow of information, from the point the patient dialed the phone to receipt of aggregate counts data by the NDP's data center, happened as follows and as shown schematically in Figure 1: Each patient call to Optum was distributed to one of its national call centers according to the availability of personnel to answer, not according to geography. Upon answering a call, the nurse would go online; call up or collect the patient's demographic information, including zip code of residence, in a new call record; and over the course of the conversation consult one or more online “guidelines.” Guidelines were electronic documents about conditions or symptoms, such as “Sore Throat/Adult” or “Influenza/Pediatric.” Whenever a nurse accessed a guideline, the guideline title would automatically be entered into the call record, serving as an indicator of the patient's symptoms. Multiple guidelines could be consulted during a single call, depending on the variety of the patient's symptoms. Each night, patient call records with guideline titles previously determined to be of interest to the NDP for syndromic surveillance purposes were extracted automatically from the Optum data system, in a uniform format specified by the NDP, to a directory accessible to software provided to Optum data-managers by the NDP. The distributed software then mapped patient calls to syndromes (e.g., respiratory), which had been previously defined with CDC collaboration, and then identified which of these represented new episodes of illness, ignoring records of patient calls in any syndrome that occurred within 42 days of an earlier call by the patient for the same syndrome. A call could be counted in multiple syndromes; for example, if the two guidelines “Blood in Stools/Pediatric” and “Cough/Pediatric” were accessed, the call would be counted in three syndromes: Gastrointestinal, Hemorrhagic, and Respiratory. A daily file was created containing counts of new episodes of each syndrome by zip code for the preceding day, and this was sent electronically in encrypted form to the NDP's data center. All extraction, processing, and transfer procedures were automated, so no extra manual data entry or active reporting was required of Optum staff.
Figure 1. Flow of telephone triage service information, from patient call to analysis.doi:10.1371/journal.pone.0005260.g001
The 24 Optum guideline titles corresponding to respiratory syndrome are listed in Table 1. This syndrome was defined broadly and, although it included cough, sore throat, and influenza, it included many other symptoms as well.
Table 1. Optum guidelines whose utilization by the nurse responding to the call would lead to a classification of respiratory syndrome.doi:10.1371/journal.pone.0005260.t001
State-level analyses were done for the 17 states with at least 500,000 inhabitants eligible to use the telephone triage services; we reasoned that states with fewer than that number of eligibles would not have robust enough call data to justify analysis. CDC weekly data on number of influenza isolates and percentage of total visits to sentinel providers for ILI were compared to the total number of calls for respiratory syndrome received in the same week. Pairwise correlations among the three data types were calculated, with lag times of 0, 1, 2, 3, and 4 weeks, using the Pearson correlation test. The number of sentinel providers and the number per 250,000 population in those 17 states were also tabulated.
Six states were selected as “representative” for graphical display: the two with the highest correlations between call data and CDC percentage ILI data (California and Wisconsin), the two with the lowest such correlations (Arizona and New Jersey), and two in the middle of the range (Missouri and Ohio). The call data presented in the graphs are rolling seven-day totals of calls for respiratory syndrome, computed day by day for the seven-day period ending on the date in question.
Influenza surveillance coordinators in the six named states were approached for permission to identify their states' data, and all consented. Coordinators in the other 11 states were not asked for such permission; each of those states is represented by a code letter in the tables.
Introducing lag-times did not increase the correlation coefficients. Correlations among the three data types with no lag are shown in Table 2, along with call service usage and CDC sentinel provider coverage. The correlation between CDC viral isolate and percentage ILI data was highest overall, with a median correlation coefficient of 0.80 (range 0.46–0.97); 12 of the 17 states (Set A: CA, WI, D, E, F, G, MO, J, K, L, M, AZ) had a coefficient of ≥0.75. The next best correlated were the telephone triage data and CDC percentage ILI data, with a median correlation coefficient of 0.74 (range 0.34–0.89) and eight states (Set B: CA, WI, D, E, F, G, H, MO) with a coefficient of ≥0.75. The median correlation between call data and CDC viral isolate data was 0.65 (range 0.35–0.83), and four states (Set C: CA, D, E, MO) had a correlation coefficient of ≥0.75 (and ≥0.80). Set C was a complete subset of both Set B and Set A. Seven of the eight states in Set B were also in Set A.
Table 2. Telephone triage service usage, number and coverage of CDC sentinel providers, and correlations among three data types, for states with at least 500,000 eligible to use the telephone triage service.doi:10.1371/journal.pone.0005260.t002
All eight states with at least one regularly reporting sentinel provider per 250,000 population had correlations between the two CDC data types of ≥0.75, i.e. were in Set A; four states with lower sentinel provider coverage also had correlations in that range. But there was no other obvious relationship between the density of sentinel providers and the correlations among any of the three data types.
The ratio of 2004 call volume to the number of people eligible to use the service ranged from 0.35% to 7.22% for the 17 states. There was no clear relationship between this usage proxy and the correlation patterns. For example, states with highest usage did not have the highest correlations between calls and sentinel provider data.
Values of the three data types over the course of the season are presented graphically for the six “representative” states in Figure 2.
Figure 2. Weekly CDC influenza isolates, CDC percentage ILI from sentinel providers, and Optum respiratory-syndrome call volume.
The selected states are the two with the highest correlations between CDC sentinel provider data and call data (California and Wisconsin), the two with the lowest correlations (Arizona and New Jersey), and the two in the middle of the range (Missouri and Ohio).doi:10.1371/journal.pone.0005260.g002
In this study of the potential utility of call data for ILI surveillance, we found considerable variation among states—even ones with adequate sentinel provider coverage—in the degree to which the call data were correlated with either CDC viral isolates or sentinel provider percent ILI data. Thus, the face validity of these call data is by no means uniformly high. There are several possible reasons, including a) insufficient coverage of the population, b) lack of specificity due to too broad a syndrome definition, c) lack of specificity due to the indirect nature of telephone diagnosis, and d) inappropriate evaluation methods. We consider each of these, with possible remedies, in turn.
a) Coverage. Population coverage of the telephone triage company described here is uneven and very sparse in many states. Although we limited the analysis to states with at least 500,000 people eligible to use the telephone triage service, it is possible that coverage and use patterns were insufficient or too heterogeneous for the data to accurately reflect ILI morbidity even in those states. A unified system that operates at the national level, like the U.K.'s universal National Health Service, with its integrated NHS Direct call system –, could provide sufficient coverage and potentially help elucidate how influenza or ILI moves across the continent . In the U.S, where telephone triage services are not widely used except in conjunction with large managed care organizations , they could be useful for surveillance at a more local level.
b) Syndrome definition. Our respiratory syndrome definition included many more conditions than ILI, such as asthma, wheezing, hoarseness, and sinus pain, and fever was not a criterion. It seems likely that using an ILI-specific syndrome definition would have improved the performance of the call data relative to the CDC data and increased its “peakedness,” helpful in determining when influenza first causes noticeable morbidity and when it peaks. Although other pathogens causing ILI, such as respiratory syncytial virus, parainfluenza, and adenovirus, also circulate in the winter months, a number of investigators have found a generally close relationship in temporal patterns between laboratory data on influenza and ILI data from clinical sources ,  as well as directly between laboratory data on influenza and ILI (“cold/flu” and fever) data from telephone triage .
c) Telephone diagnosis vs. direct clinical diagnosis. One might expect an unacceptable loss of specificity in using telephone health data, but several studies have found that call data and clinical data are closely related for ILI and certain other syndromes , , , , and it has been argued on the basis of such results that calls are a “timely, useful and representative data stream that shows promise for integration into a real-time syndromic surveillance system” .
d) Evaluation methods. It is not self-evident that the degree of correlation with the CDC surveillance data is the best measure of utility of the telephone triage data for ILI surveillance. Other comparative approaches have been used or suggested (especially to compare timeliness of data sources), including cross-correlation time series modeling , , comparison of peaks , and comparison of aberration detection . On reviewing the literature on influenza surveillance, Dailey et al.  concluded that an aberration detection method (e.g., use of a threshold, CUSUM, or scan statistic) was preferable to both comparison of peaks and correlation. In previous unpublished work, we have not found the application of scan statistics to syndromic ILI data to be informative. However, a method in which a threshold for percentage of encounters for ILI is established is appealing; Cooper et al.  demonstrated that age-group-specific thresholds of percentage of total calls that are for “cold/flu” syndrome, derived by means of Poisson regression models, were useful, providing 6–14 days of advance warning of seasonal influenza activity.
In synthesis, in the absence of a universal health care system, telephone triage data, even when from a single company with national scope, will be patchy in coverage and therefore not a reliable source of ILI surveillance data on a national scale. However, in states with higher correlation between the call data and CDC sentinel surveillance percentage ILI and/or virologic data (and a sizeable number of residents eligible to use a telephone triage service), call data may serve as a useful adjunct to CDC influenza/ILI surveillance data, for example at times when sentinel surveillance is not in operation or in areas where sentinel provider coverage is considered insufficient. Utility for ILI surveillance is likely to be higher if an ILI-specific syndrome definition can be employed in the electronic data collection. A threshold method of aberration detection seems most promising for near-real-time ILI surveillance. The relative advantage to public health of analyzing telephone triage data as a separate, potentially more timely data stream vs. combining these data with ED or ambulatory care data for a larger, possibly more robust source remains to be determined.
The authors wish to thank the programmers and analysts at Optum and at the NDP data center.
Conceived and designed the experiments: AA KK MK RP RP. Analyzed the data: AA. Contributed reagents/materials/analysis tools: KST SW. Wrote the paper: WKY KST. Contributed substantially to the interpretation of the data: WKY. Was the main author of the manuscript: WKY. Provided data: KST SW. Wrote pieces of the manuscript: KST. Contributed to the design of the study, conducted the analyses, and displayed the results graphically: AA. Critically reviewed the manuscript: AA RP RH SW. Was centrally involved in the design of the study and interpretation of the data and provided critical comments on the manuscript: KK MK. Contributed to the conception of the study: RP. Facilitated the acquisition of data: RP RH. Conceived of the study, was centrally involved in its design and in the interpretation of data, and provided critical comments on the manuscript: RP.
- 1. Thompson W, Shay D, Weintraub E, Brammer L, Cox N, et al. (2003) Mortality associated with influenza and respiratory syncytial virus in the United States. JAMA 289: 179–186.
- 2. Meltzer MI, Cox NJ, Fukuda K (1999) The economic impact of pandemic influenza in the United States: priorities for intervention. Emerg Infect Dis 5: 659–671.
- 3. Centers for Disease Control and Prevention (2002) Surveillance for influenza—United States, 1997–98, 1998–99, and 1999–00 seasons. Morb Mortal Wkly Rep Surveill Summ 51(7): 1–10.
- 4. Lazarus R, Kleinman K, Dashevsky I, DeMaria A, Platt R (2001) Using automated medical records for rapid identification of illness syndromes: the example of lower respiratory infection. BMC Public Health 1: 1–9.
- 5. Lewis MD, Pavlin JA, Mansfield JL, O'Brien S, Boomsma LG, et al. (2002) Disease outbreak detection system using syndromic data in the greater Washington DC area. Am J Prev Med 23: 180–186.
- 6. Miller B, Kassenborg H, Dunsmuir W, Griffith J, Hadidi M, et al. (2004) Syndrome surveillance for influenzalike illness in ambulatory care setting. Emerg Infect Dis 10: 1806–1811.
- 7. Heffernan R, Mostashari F, Das D, Karpati A, Kulldorff M, et al. (2004) Syndromic surveillance in public health practice: the New York City emergency department system. Emerg Infect Dis 10: 858–864.
- 8. Kulldorff M, Heffernan R, Hartman J, Assunção R, Mostashari F (2005) A space-time permutation scan statistic for the early detection of disease outbreaks. PLoS Medicine 2: 216–224.
- 9. Ritzwoller DP, Kleinman K, Palen TE, Abrams A, Kaferly J, et al. (2005) Comparison of syndromic surveillance and a sentinel provider system in detecting the 2003 Denver influenza outbreak. Morb Mortal Wkly Rep 54: (suppl)151–156.
- 10. Brownstein JS, Kleinman KP, Mandl KD (2005) Identifying pediatric age groups for influenza vaccination using a real-time regional surveillance system. Am J Epidemiol 162: 686–693.
- 11. Clothier HJ, Atkin L, Turner J, Sundararajan V, Kelly HA (2006) A comparison of data sources for the surveillance of seasonal and pandemic influenza in Victoria. Commun Dis Intell 30: 345–349.
- 12. Harcourt SE, Smith GE, Hollyoak V, Joseph CA, Chaloner R, et al. (2001) Can calls to NHS Direct be used for syndromic surveillance? Commun Dis Public Health 4: 178–182.
- 13. Cooper DL, Smith GE, Hollyoak VA, Joseph CA, Johnson L, et al. (2002) Use of NHS Direct calls for surveillance of influenza—a second year's experience. Commun Dis Public Health 5: 127–131.
- 14. Doroshenko A, Cooper D, Smith G, Gerard E, Chinemana F, et al. (2005) Evaluation of syndromic surveillance data based on National Health Service Direct derived data—England and Wales. Morb Mortal Wkly Rep 54: (suppl)117–122.
- 15. Smith GE, Cooper DL, Loveridge P, Chinemana F, Gerard E, et al. (2006) A national syndromic surveillance system for England and Wales using calls to a telephone helpline. Euro Surveill 11: 220–224.
- 16. Moore K (2004) Real-time syndromic surveillance in Ontario, Canada: the potential use of emergency departments and Telehealth. Eur J Emerg Med 11: 3–11.
- 17. van Dijk A, McGuinness D Rolland E, Moore KM (2008) Can Telehealth Ontario respiratory call volume be used as a proxy for emergency department respiratory visit surveillance by public health? Canadian J Emerg Med 10: 18–24.
- 18. Cooper DL, Smith GE, Edmunds WJ, Joseph C, Gerard E (2007) The contribution of respiratory pathogens to the seasonality of NHS Direct calls. J Infect 55: 240–248.
- 19. Centers for Disease Control and Prevention (2005) Update: influenza activity—United States and worldwide, 2004–05 season. Morb Mortal Wkly Rep 54: 631–634.
- 20. Platt R, Bocchino C, Caldwell B, Harmon R, Kleinman K, et al. (2003) Syndromic surveillance using minimum transfer of identifiable data: the example of the National Bioterrorism Syndromic Surveillance Demonstration Program. J Urban Health 80: (Suppl. 1)i25–i31.
- 21. Yih WK, Caldwell B, Harmon R, Kleinman K, Lazarus R, et al. (2004) The National Bioterrorism Syndromic Surveillance Demonstration Program. In: Syndromic Surveillance: Reports from a National Conference, 2003. Morb Mortal Wkly Rep 53: (suppl)43–46.
- 22. Paget J, Marquet R, Meijer A, van der Velden K (2007) Influenza activity in Europe during eight seasons (1999–2007): an evaluation of the indicators used to measure activity and an assessment of the timing, length and course of peak activity (spread) across Europe. BMC Infect Dis 7: 141.
- 23. Espino JU, Hogan WR, Wagner MM (2003) Telephone triage: a timely data source for surveillance of influenza-like diseases. Am Med Inform Assoc Annu Symp Proc 2003: 215–219.
- 24. Zheng W, Aitken R, Muscatello DJ, Churches T (2007) Potential for early warning of viral influenza activity in the community by monitoring clinical diagnoses of influenza in hospital emergency departments. BMC Public Health 7: 250.
- 25. Henry JV, Magruder S, Snyder M (2004) Comparison of office visit and nurse advice hotline data for syndromic surveillance—Baltimore-Washington, D.C., 2002. Morb Mortal Wkly Rep 53: (suppl)112–116.
- 26. Dailey L, Watkins RE, Plant AJ (2007) Timeliness of data sources used for influenza surveillance. J Am Med Inform Assoc 14: 626–631.