Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

The Evidence Base for Interventions Delivered to Children in Primary Care: An Overview of Cochrane Systematic Reviews

  • Peter J. Gill ,

    peter.gill@phc.ox.ac.uk

    Affiliations Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom, Faculty of Medicine and Dentistry, University of Alberta, Edmonton, Canada

  • Kay Yee Wang,

    Affiliation Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

  • David Mant,

    Affiliation Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

  • Lisa Hartling,

    Affiliations Department of Pediatrics, Alberta Research Centre for Health Evidence, University of Alberta, Edmonton, Canada, Department of Pediatrics, Cochrane Child Health Field, University of Alberta, Edmonton, Canada

  • Carl Heneghan,

    Affiliation Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

  • Rafael Perera,

    Affiliation Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

  • Terry Klassen,

    Affiliations Department of Pediatrics, Cochrane Child Health Field, University of Alberta, Edmonton, Canada, Department of Pediatrics and Child Health, Manitoba Institute of Child Health, University of Manitoba, Winnipeg, Canada

  • Anthony Harnden

    Affiliation Department of Primary Care Health Sciences, University of Oxford, Oxford, United Kingdom

Abstract

Background

As a first step in developing a framework to evaluate and improve the quality of care of children in primary care there is a need to identify the evidence base underpinning interventions relevant to child health. Our objective was to identify all Cochrane systematic reviews relevant to the management of childhood conditions in primary care and to assess the extent to which Cochrane reviews reflect the burden of childhood illness presenting in primary care.

Methodology/Principal Findings

We used the Cochrane Child Health Field register of child-relevant systematic reviews to complete an overview of Cochrane reviews related to the management of children in primary care. We compared the proportion of systematic reviews with the proportion of consultations in Australia, US, Dutch and UK general practice in children. We identified 396 relevant systematic reviews; 358 included primary studies on children while 251 undertook a meta-analysis. Most reviews (n = 218, 55%) focused on chronic conditions and over half (n = 216, 57%) evaluated drug interventions. Since 2000, the percentage of pediatric primary care relevant reviews only increased by 2% (7% to 9%) compared to 18% (10% to 28%) in all child relevant reviews. Almost a quarter of reviews (n = 78, 23%) were published on asthma treatments which only account for 3–5% of consultations. Conversely, 15–23% of consultations are due to skin conditions yet they represent only 7% (n = 23) of reviews.

Conclusions/Significance

Although Cochrane systematic reviews focus on clinical trials and do not provide a comprehensive picture of the evidence base underpinning the management of children in primary care, the mismatch between the focus of the published research and the focus of clinical activity is striking. Clinical trials are an important component of the evidence base and the lack of trial evidence to demonstrate intervention effectiveness in substantial areas of primary care for children should be addressed.

Introduction

In both developed and developing countries, the clinical care of children accounts for a large proportion of primary care. For example, in the UK, over 95% of NHS clinical contacts are made in general practice with over 300 million taking place each year,[1] and consultations with children under 15 years represent 18% of all primary care consultations in the UK, 17% in the US and 12% in Australia [2], [3], [4]. A 10,000-patient general practice will expect to have 1,500 children under 16 years registered [2].

There are concerns about the quality of primary care delivered to children in the UK due to episodes highlighted by the confidential enquiry into child deaths and increases in Accident and Emergency presentations, short admissions and unplanned hospitalizations [5], [6]. Similar concerns about the quality of ambulatory care for children have been documented in the US [7]. Globally, more than 10.6 million children under 5 years of age die each year, mostly from preventable and treatable causes, including pneumonia, diarrhea, malaria, injuries, HIV/AIDS, measles and malnutrition [8], [9], [10], [11]. The high level of child mortality in poor countries indicates that high quality evidence based provision of child healthcare is an even greater problem.

However, measuring the quality of care of children is challenging due to the lack of pediatric quality measures [12]. The first step to developing a framework to evaluate and improve the quality of care of children is the identification of the evidence base for interventions relevant to child health and primary care. Systematic reviews are the most comprehensive syntheses of information in healthcare [13] and represent a logical starting point to identify high quality evidence relevant to a broad clinical topic.

The number of systematic reviews published annually has steadily increased over the past decade [14]. In 2007 alone, 2,500 systematic reviews of research were published, with roughly 20% of them produced through the Cochrane Database of Systematic Reviews (CDSR) [15]. The primary aim of the Cochrane Collaboration is to make available up-to-date syntheses of reliable evidence regarding the benefits and risks of healthcare interventions [16]. Cochrane systematic reviews are widely regarded as methodologically rigorous and of higher quality than non-Cochrane systematic reviews [16], [17], [18], [19], [20]. The CDSR therefore represents a sample of high quality studies to better understand the current state of the evidence base.

A previous review described the evidence available from child-relevant systematic reviews in the CDSR, yet it did not determine the evidence base relevant to primary care [16]. In addition no previous studies have compared the scope and number of systematic reviews published with the burden of illness in general practice. Therefore, we completed an overview of systematic reviews in the CDSR relevant to the management of childhood conditions in primary care and compared the topics and number of systematic reviews with the burden of childhood illness in primary care. These efforts provide a framework for researchers and policy makers to identify avoidable waste in the production of research evidence [15] and will lead to further initiatives in the CDSR and beyond to improve the quality of evidence for children in primary care.

Methods

Ethics

Data for this study was acquired through previously published work, no patient or hospital data was accessed. Therefore, written consent and institutional ethical review was not required for this research. The PRISMA checklist and flow diagram are available as supporting information; see PRISMA Checklist S1 and PRISMA Flow Diagram S1.

Searching

We identified eligible systematic reviews from the Cochrane Child Health Field Reviews Register, a register of all child health relevant systematic reviews published in the CDSR. The search terms used to develop the register are outlined in Appendix S1. The register includes all Cochrane systematic reviews that intended to include children (0–18 years of age) or studied an intervention intended to improve the health and wellbeing of children, consistent with those of the Cochrane Child Health Field Trials Register [16], including reviews that included both children and adult participants. The original register included all studies identified in Issue 2, 2009 of the CDSR and was updated in Issue 8, 2010. We did not include Cochrane protocols.

Study selection

Three reviewers (PG, KW and AH) independently screened a sample of 100 abstracts to pilot and refine the inclusion criteria. Subsequently, two reviewers (PG and KW) independently screened all potentially relevant studies for inclusion. Disagreement was resolved by consensus or consultation with a third reviewer (AH or DM) at all stages. We included systematic reviews regardless of the study designs included. We excluded studies that focused only on children aged less than one month as we felt that the majority of neonatal interventions would be delivered in secondary and tertiary care.

Our definition of ‘primary care setting’ included general or family practice, ambulatory care, pediatric outpatient clinics, pediatric assessment units or emergency departments. We included systematic reviews that were relevant to the management of childhood conditions in primary care or which evaluated interventions that could be administered in the primary care setting by a healthcare provider (screening for early detection of disease, diagnosis of conditions, initiation of treatment, referral for secondary care and ongoing monitoring in primary care). To focus on topics specific to general practice, we excluded school based interventions, specialist nurses, interventions delivered during pregnancy, public health programs and orthodontic and specialist dental procedures.

Data abstraction

The Cochrane Child Health Field provided data from systematic review register in a Microsoft Excel sheet. The data available in the register for each systematic review are described elsewhere [16] and include general review characteristics, characteristics of included studies and methodological approaches. We updated the register and classified included systematic reviews according to type of care (i.e. acute, chronic or preventative) [7] and applicability to developing or developed countries (defined by the United Nations, which includes Europe, Canada, the USA, Australia, New Zealand, and Japan). One reviewer (PG) extracted data which were checked by a second (KW). Disagreement was resolved by consensus or consultation with a third reviewer (AH or DM) at all stages.

Burden of illness

We identified burden of illness data from publically available datasets for Australia, the Netherlands, the UK and the US which were classified according to the International Classification of Disease revision 9 (ICD-9) or the International Classification of Primary Care 2nd Edition (ICPC-2). In Australia, the Bettering the Evaluation and Care of Health (BEACH) Project provides an annual database of approximately 100,000 general practice records classified by ICPC-2 [21]. The Royal College of General Practitioners Weekly Returns Service (WRS) is a sentinel general practice network located across England and Wales that routinely monitors disease episodes from every consultation on a weekly basis by ICD-9 [2], [22], [23]. In the US, the Centers for Disease Control and Prevention and the National Center for Health Statistics publish periodic reports on ambulatory healthcare visits from specific time periods [3]. Lastly, we obtained Dutch data on childhood morbidity in general practice through a literature search [24]. The ages included in the dataset ranged from 0–15 to 0–17 years. Where possible, we used estimates over a 3-year period to reduce the possibility of year-to-year fluctuations. The codes reflect primary diagnosis (US), last diagnosis in consultation (the Netherlands) or any diagnosis (Australia and UK) [2], [3], [21], [24], [25]. Estimates of the burden of illness data are approximate rather than precise due to the limitations of the disease coding systems and availability of data. Further details are outlined in Appendix S2.

ICD-9 has many limitations in providing useful clinical data in primary care due to the structure of the axes (e.g. diagnostic entities can with equal logic be classified in more than one chapter) [26],[27]. Several specific diagnostic codes must be combined to generate clinically meaningful data (see Appendix S2). ICPC-2 was developed for use in primary care and classifies clinical activity in the domains of General/Family Practice, taking into account the patient's reason for encounter, the problems/diagnosis managed and interventions [28]. We categorized all systematic reviews by ICPC-2 and by ICD-9 Disease Related Groups. When comparing systematic reviews to burden of illness data, we excluded reviews relevant to developing countries only, as the burden of illness data is specific to developed countries. Due to the structural differences in coding systems, we present the findings separately by ICPC-2 chapters and by ICD-9 (specific diagnostic categories) where relevant. We also compared the number of systematic reviews relevant to low-income countries with the major causes of death in children under 5 years of age other than neonates in Africa and Southeast Asia [10]. Data were analyzed descriptively in PASW Statistics Version 18.0 using frequencies and proportions.

Results

Flow of included studies

From 1,183 systematic reviews we excluded 787 systematic reviews that did not fulfill the inclusion criteria (Figure 1). We included 396 systematic reviews; a full list of included systematic reviews is included in Table S1.

Study characteristics

Of the 396 systematic reviews, 38 (10%) found no relevant studies (n = 22) or only found studies whose participants did not include children (n = 16) (Table 1). There were 216 systematic reviews (57%) of drug interventions and 14 (4%) which evaluated a drug and a complex or surgical intervention. Fifty-five (14%) systematic reviews focused exclusively on developing countries. Of the 6,022 primary studies included in the reviews, 3,385 included children only and 899 included children and adults.

thumbnail
Table 1. Characteristics of included Cochrane systematic reviews.

https://doi.org/10.1371/journal.pone.0023051.t001

A greater percentage of systematic reviews on complex interventions (n = 9, 24%) and natural health products (n = 8, 21%) found no relevant studies or children participants compared to included reviews (complex, n = 43, 11%; natural health product, n = 34, 9%). Of the 38 reviews that did not find any published studies, the main conditions studied were: respiratory (n = 10), blood and immune mechanism (n = 6), psychological (n = 6), digestive (n = 3) and neurological (n = 3).

The first systematic review was published in 1996 and since 2003, the rate of new publications has remained relatively constant at a median of 37 per year (range: 28–44). Nearly half of the systematic reviews (44%) were updated in 2008 or more recently, up-to-date as defined by the Cochrane Collaboration. Of reviews considered out-of-date, two were last updated in 1998.

Figure 2 illustrates the increase in the number of systematic reviews in the CDSR relevant to pediatric primary care compared to the increase in all child relevant and all Cochrane reviews. In 2009, there were 4,005 Cochrane reviews, of which 1,110 (28%) were on children and 370 (9%) were relevant to pediatric primary care. Since 2000, the percentage of child relevant reviews nearly tripled (from 10% to 28%) compared to a much smaller increase of 2% (from 7% to 9%) in pediatric primary care relevant reviews.

thumbnail
Figure 2. The cumulative number of systematic reviews published in the Cochrane Database of Systematic Reviews, 1995 to 2009.

https://doi.org/10.1371/journal.pone.0023051.g002

Figure 3 illustrates systematic reviews categorized using the International Classification of Primary Care 2nd Edition. Of note, five conditions cover 76% of all systematic reviews: respiratory (37%, n = 148), psychological (11%, n = 42), general and unspecified (e.g. fever; 10%, n = 41), digestive (10%, n = 40), and neurological (7%, n = 29).

thumbnail
Figure 3. Classification of Cochrane systematic reviews relevant to children in primary care.

All Cochrane systematic reviews relevant to children in primary care were classified according to the International Classification of Primary Care 2nd Edition.

https://doi.org/10.1371/journal.pone.0023051.g003

Consultations in primary care

Table 2 shows the most common reasons for childhood consultations in Australian and Dutch general practice, excluding systematic reviews relevant to developing countries only (n = 55). Respiratory conditions are the most common reason for general practice consultations in children representing 23–32% of all consultations, of which 43% (n = 147) of included systematic reviews are relevant. There was a discrepancy between the burden of illness and the corresponding number of systematic reviews. For instance, despite representing over 15% of general practice consultations, only 7% (n = 23) of systematic reviews were relevant to skin conditions. In contrast, 12% (n = 42) of systematic reviews were on psychological conditions even though these only represent a relatively small proportion of consultations in Australian and Dutch general practice (2–3%).

thumbnail
Table 2. Comparison of childhood consultations in Australian and Dutch general practice with number of systematic reviews.

https://doi.org/10.1371/journal.pone.0023051.t002

Table 3 evaluates specific diagnostic categories in greater detail and shows that despite asthma representing 3–5% of consultations in Australia, UK and US, it is the focus of 23% (n = 78) of systematic reviews. However, only 2% (n = 8) of systematic reviews were on injuries despite being the reason for 7–10% of consultations.

thumbnail
Table 3. Comparison of reasons for childhood consultations in Australia, UK and US with number of systematic reviews.

https://doi.org/10.1371/journal.pone.0023051.t003

Major causes of under-five mortality

Table 4 compares the major causes of death in children younger than age 5 years in Africa and Southeast Asia and shows that while certain topics have a large proportion of systematic reviews, such as HIV/AIDS (22%, n = 12) and diarrheal diseases (22%, n = 12), childhood injuries and pneumonia have a relative lack of Cochrane reviews.

thumbnail
Table 4. Comparison of the major causes of death in children younger than age 5 years other than neonates in Africa and Southeast Asia with number of relevant systematic reviews.

https://doi.org/10.1371/journal.pone.0023051.t004

Discussion

Statement of principal findings

Our study is the first comprehensive mapping of the applicability of Cochrane systematic reviews to a large clinical topic – children in primary care – including approximately 10% of all reviews published in the Cochrane database. Although previous studies have shown that Cochrane systematic reviews only represent 20% of all published reviews [15], they are the principal international source of quality-assessed evidence on clinical effectiveness for health care workers and policy decision makers.

Our findings show some striking mismatches between the conditions for which the Cochrane database provides clinical evidence and the burden of childhood illness in primary care. For example, despite the increasing incidence of skin conditions (such as rash, eczema and impetigo) which precipitate up to 23% of all primary care consultations [21], [24], [29], only 7% of included Cochrane reviews are relevant to this topic. In contrast, 23% of reviews were on asthma which accounts for only 3–5% of clinical consultations. There are no similar primary care consultation data for low-income countries, but while the importance of HIV and malaria as important causes of childhood mortality is recognized in the Cochrane database, there is a relative lack of Cochrane review evidence on pneumonia and childhood injuries. Both conditions are commonly seen in primary care in low-income countries and are associated with high mortality [10].

Possible explanations for the mismatch with burden of illness in primary care

There are several potential reasons for the observed discrepancy in the number of Cochrane reviews and the burden of consultations. First, systematic reviews are based on primary research and the deficiencies may reflect the lack of primary studies available for synthesis. For example, reviews of complex interventions and natural health products were more likely to identify studies with adult participants only or find no relevant studies. The lack of controlled trials in child health is well documented and due to numerous reasons [30]. Second, clinicians and researchers identify topics for Cochrane systematic reviews based on clinical uncertainty, practice variation or research interests. These may not correlate with the number of consultations or burden of illness. Third, public funding of research is correlated only modestly with disease burden, if at all [15]. Fourth, there may be more interventions (and comparisons) available for certain conditions than others. Fifth, certain areas may be broader in scope and represent more varied conditions (e.g. neurological disorders) that warrant a disproportionately greater volume of research relative to their consultation burden. Finally, the lack of pediatric training for general practitioners in many countries and therefore lack of academic interest in pediatric primary care is likely to contribute to the lack of research endeavor.

Relevance to previous research

Previous studies have suggested that there may be insufficient evidence specific to children in certain topic areas [13], [31], [32], [33], especially non-drug interventions such as counseling and advice [34]. The emphasis on drug interventions is reflected in the funding of both commercial research and non-commercial research [35] while 69% of all pediatric controlled trials assess drug products [30]. In our study, we also found an over-representation of reviews relevant to drug interventions. Non-drug interventions are an important part of primary care and need to be reflected in evidence syntheses and research funding.

It is essential that systematic reviews address questions that are relevant to patients, clinicians and policy makers [14]. Several research groups have attempted to quantify the evidence that is available and used by physicians making treatment decisions [36], [37]. Even when reviews are applicable to children in general practice often authors fail to provide a description of how the intervention can be applied in clinical practice, reinforcing the findings of previous studies [38], [39]. Current work is being completed to determine how primary care practitioners use and apply Cochrane reviews in practice.

Methodological limitations in the conduct of systematic reviews

Several of the systematic reviews in our study that used a meta-analysis included both children and adults; this observation is consistent with other studies [13], [16]. Combining studies with adults and children does not identify potentially important differences in efficacy and safety between these population groups [13] and can result in ineffective or even unsafe medical care [13], [40]. To derive relevant evidence for decision-making, systematic reviews should consider potentially important differences in effects across different populations. Guidance to inform subgroup analyses is currently being prepared through StaR Child Health (www.ifsrc.org), an initiative aimed at improving the design, conduct, and reporting of pediatric research [40].

Cochrane systematic reviews are expected to be updated every two years, [41] yet 56% of systematic reviews were last assessed as up-to-date prior to 2008 (as of August 2010), greater than 38% reported elsewhere [16]. Several authors have reiterated the importance of developing mechanisms and channeling resources to ensure that systematic reviews are updated as new evidence emerges [14], [42].

Limitations of the study

Our definition of primary care setting included pediatric assessment units and emergency departments, which may have partially explained the discrepancy between consultations and reviews. However, we only included studies that could be administered in the primary care setting by a healthcare provider. Our analysis was restricted to Cochrane reviews of interventions; there may be systematic reviews published outside the CDSR that fill some of the clinical gaps identified, particularly screening, diagnostic and prognostic reviews. In addition, we did not assess the quality of included systematic reviews. Classification of the reviews by ICPC-2 and ICD-9 has several limitations and was completed to provide an approximate comparison rather than precise figures.

Implications for research in children in primary care

The study highlights the on-going challenges of evidence based practice in primary care and the importance of characterizing the current evidence base. Further steps are required to identify and prioritize which systematic reviews are needed in primary care and encourage Cochrane Review Groups, funders, and other relevant organizations to promote those topics among potential authors. The recently implemented initiative to register all systematic review protocols is a step forward [43], [44]. There is a need for clinicians, researchers and policy makers in primary care to improve the evidence base in children by determining if the current evidence base adequately informs clinical practice.

Supporting Information

Appendix S1.

Search terms used by the Cochrane Child Health Field. The search strategy used to develop the Cochrane Child Health Field Reviews Register, a register of all child health relevant systematic reviews published in the CDSR.

https://doi.org/10.1371/journal.pone.0023051.s001

(DOC)

Appendix S2.

Methodology of classifying systematic reviews by burden of illness data.

https://doi.org/10.1371/journal.pone.0023051.s002

(DOC)

Table S1.

List of all included Cochrane systematic reviews.

https://doi.org/10.1371/journal.pone.0023051.s003

(DOC)

Acknowledgments

Ethics approval was not required for the conduct of this systematic review. We would like to thank Denise Thomson for providing access to the systematic reviews registry, Mike Clarke for assistance in developing the study question and Iain Chalmers for his invaluable input on the manuscript.

Author Contributions

Conceived and designed the experiments: PG KW DM LH CH RP TK AH. Performed the experiments: PG KW DM CH AH. Analyzed the data: PG KW DM CH RP AH. Wrote the paper: PG KW DM LH CH RP TK AH. Provided input on the data analysis: KW DM LH CH RP TK AH. Provided clinical advice: KW DM CH TK AH. Provided methodological advice: LH CH TK. Provided critical revision of the manuscript for important intellectual content: KW DM LH CH RP TK AH.

References

  1. 1. The Academy of Medical Sciences (2009) Research in general practice: bringing innovation into patient care. London: The Academy of Medical Sciences.
  2. 2. Royal College of General Practitioners (2008) Weekly Returns Service Annual Prevalence Report 2007. Birmingham Research Unit. pp. 1–89.
  3. 3. Freid VM, Makuc DM, Rooks RN (1998) Ambulatory health care visits by children: principal diagnosis and place of visit. Vital Health Stat 13: 1–23.
  4. 4. Britt H, Miller GC, Charles J, Bayram C, Pan Y, et al. (2008) General practice activity in Australia 2006–07. Cat. no. GEP 21. Canberra: Australian Institute of Health and Welfare.
  5. 5. Harnden A, Mayon-White R, Mant D, Kelly D, Pearson G (2009) Child deaths: confidential enquiry into the role and quality of UK primary care. Br J Gen Pract 59: 819–824.
  6. 6. Saxena S, Bottle A, Gilbert R, Sharland M (2009) Increasing Short-Stay Unplanned Hospital Admissions among Children in England; Time Trends Analysis ′97-′06. PLoS ONE 4: e7484.
  7. 7. Mangione-Smith R, DeCristofaro AH, Setodji CM, Keesey J, Klein DJ, et al. (2007) The quality of ambulatory care delivered to children in the United States. N Engl J Med 357: 1515–1523.
  8. 8. United Nations Children's Fund (2007) The state of the world's children 2008. New York.
  9. 9. Loaiza E, Wardlaw T, Salama P (2008) Child mortality 30 years after the Alma-Ata Declaration. Lancet 372: 874–876.
  10. 10. Bryce J, Boschi-Pinto C, Shibuya K, Black RE (2005) WHO estimates of the causes of death in children. Lancet 365: 1147–1152.
  11. 11. World Health Organization (WHO) (2004) The global burden of disease: 2004 update.
  12. 12. Beal AC, Co JP, Dougherty D, Jorsling T, Kam J, et al. (2004) Quality measures for children's health care. Pediatrics 113: 199–209.
  13. 13. Cramer K, Wiebe N, Moyer V, Hartling L, Williams K, et al. (2005) Children in reviews: Methodological issues in child-relevant evidence syntheses. BMC Pediatrics 5: 38.
  14. 14. Bastian H, Glasziou P, Chalmers I (2010) Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 7: e1000326.
  15. 15. Chalmers I, Glasziou P (2009) Avoidable waste in the production and reporting of research evidence. Lancet 374: 86–89.
  16. 16. Bow S, Klassen J, Chisholm A, Tjosvold L, Thomson D, et al. (2010) A descriptive analysis of child-relevant systematic reviews in the Cochrane Database of Systematic Reviews. BMC Pediatr 10: 34.
  17. 17. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG (2007) Epidemiology and reporting characteristics of systematic reviews. PLoS Med 4: 447–455.
  18. 18. Wen J, Ren Y, Wang L, Li Y, Liu Y, et al. (2008) The reporting quality of meta-analyses improves: a random sampling study. J Clin Epidemiol 61: 770–775.
  19. 19. Moseley AM, Elkins MR, Herbert RD, Maher CG, Sherrington C (2009) Cochrane reviews used more rigorous methods than non-Cochrane reviews: survey of systematic reviews in physiotherapy. J Clin Epidemiol 62: 1021–1030.
  20. 20. Jadad AR, Cook DJ, Jones A, Klassen TP, Tugwell P, et al. (1998) Methodology and reports of systematic reviews and meta-analyses: a comparison of Cochrane reviews with articles published in paper-based journals. JAMA 280: 278–280.
  21. 21. Charles J, Pan Y, Britt H (2004) Trends in childhood illness and treatment in Australian general practice, 1971-2001. Med J Aust 180: 216–219.
  22. 22. Fleming DM (1999) Weekly Returns Service of the Royal College of General Practitioners. Commun Dis Public Health 2: 96–100.
  23. 23. Fleming DM, Miles J (2010) The representativeness of sentinel practice networks. J Public Health (Oxf) 32: 90–96.
  24. 24. Otters HB, van der Wouden JC, Schellevis FG, van Suijlekom-Smit LW, Koes BW (2005) Changing morbidity patterns in children in Dutch general practice: 1987-2001. Eur J Gen Pract 11: 17–22.
  25. 25. Britt H, Miller G, Bayram C (2007) The quality of data on general practice - a discussion of BEACH reliability and validity. Aust Fam Physician 36: 36–40.
  26. 26. World Health Organization (2011) International Classification of Diseases (ICD). Available: http://www.who.int/classifications/icd/en/. Accessed March 30, 2011.
  27. 27. Soler JK, Okkes I, Wood M, Lamberts H (2008) The coming of age of ICPC: celebrating the 21st birthday of the International Classification of Primary Care. Fam Pract 25: 312–317.
  28. 28. World Health Organization (2011) International Classification of Primary Care, Second edition (ICPC-2). Available: http://www.who.int/classifications/icd/adaptations/icpc2/en/index.html. Accessed March 30, 2011.
  29. 29. Bruijnzeels M, Foets M, van der Wouden J, van den Heuvel W, Prins A (1998) Everyday symptoms in childhood: occurrence and general practitioner consultation rates. Br J Gen Pract 48: 880–884.
  30. 30. Thomson D, Hartling L, Cohen E, Vandermeer B, Tjosvold L, et al. (2010) Controlled trials in children: quantity, methodological quality and descriptive characteristics of pediatric controlled trials published 1948-2006. PLoS ONE 5:
  31. 31. Christensen ML, Helms RA, Chesney RW (1999) Is pediatric labeling really necessary? Pediatrics 104: 593–597.
  32. 32. Klassen TP, Hartling L, Craig JC, Offringa M (2008) Children are not just small adults: the urgent need for high-quality trial evidence in children. PLoS Med 5: e172.
  33. 33. Martinez-Castaldi C, Silverstein M, Bauchner H (2008) Child versus adult research: the gap in high-quality study design. Pediatrics 122: 52–57.
  34. 34. Rudolf MC, Lyth N, Bundle A, Rowland G, Kelly A, et al. (1999) A search for the evidence supporting community paediatric practice. Arch Dis Child 80: 257–261.
  35. 35. Chalmers I, Rounding C, Lock K (2003) Descriptive survey of non-commercial randomised controlled trials in the United Kingdom, 1980-2002. BMJ 327: 1017.
  36. 36. Gill P, Dowell AC, Neal RD, Smith N, Heywood P, et al. (1996) Evidence based general practice: a retrospective study of interventions in one training practice. BMJ 312: 819–821.
  37. 37. Waters KL, Wiebe N, Cramer K, Hartling L, Klassen TP (2006) Treatment in the pediatric emergency department is evidence based: a retrospective analysis. BMC Pediatr 6: 26.
  38. 38. Glasziou P, Meats E, Heneghan C, Shepperd S (2008) What is missing from descriptions of treatment in trials and reviews? BMJ 336: 1472–1474.
  39. 39. Glasziou P, Chalmers I, Altman DG, Bastian H, Boutron I, et al. (2010) Taking healthcare interventions from trial to practice. BMJ 341: c3852.
  40. 40. Klassen TP, Hartling L, Hamm M, van der Lee JH, Ursum J, et al. (2009) StaR Child Health: an initiative for RCTs in children. Lancet 374: 1310–1312.
  41. 41. Higgins JPT, Green S, editors. (2008) Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration. Available: www.cochrane-handbook.org.
  42. 42. Garritty C, Tsertsvadze A, Tricco AC, Sampson M, Moher D (2010) Updating systematic reviews: an international survey. PLoS ONE 5: e9914.
  43. 43. Straus S, Moher D (2010) Registering systematic reviews. CMAJ 182: 13–14.
  44. 44. Booth A, Clarke M, Ghersi D, Moher D, Petticrew M, et al. (2010) An international registry of systematic-review protocols. Lancet 377: 108–109.