Figures
Abstract
Background
Clinical practice guidelines are important for guiding practice, but it is unclear if they are commensurate with the available evidence.
Methods
We examined guidelines produced by cancer and gynecological societies and organizations and evaluated their coverage of and stance towards chemotherapy for advanced stage disease among 4 gynecological malignancies (breast, ovarian, cervical, endometrial cancer) where the evidence for the use of chemotherapy is very different (substantial and conclusive for breast and ovarian cancer, limited and suggesting no major benefit for cervical and endometrial cancer). Eligible societies and organizations were identified through systematic internet searches (last update June 2009). Pertinent websites were scrutinized for presence of clinical practice guidelines, and relative guidelines were analyzed.
Results
Among 224 identified eligible societies and organizations, 69 (31%) provided any sort of guidelines, while recommendations for chemotherapy on advanced stage gynecological malignancies were available in 20 of them. Only 14 had developed their own guideline, and only 5 had developed guidelines for all 4 malignancies. Use of levels of evidence and grades of recommendations, and aspects of the production, implementation, and timeliness of the guidelines did not differ significantly across malignancies. Guidelines on breast and ovarian cancer utilized significantly more randomized trials and meta-analyses. Guidelines differed across malignancies on their coverage of disease-free survival (p = 0.033), response rates (p = 0.024), symptoms relief (p = 0.005), quality of life (p = 0.001) and toxicity (p = 0.039), with breast and ovarian cancer guidelines typically covering more frequently these outcomes. All guidelines explicitly or implicitly endorsed the use of chemotherapy.
Citation: Polyzos NP, Mauri D, Ioannidis JPA (2011) Guidelines on Chemotherapy in Advanced Stage Gynecological Malignancies: An Evaluation of 224 Professional Societies and Organizations. PLoS ONE 6(5): e20106. https://doi.org/10.1371/journal.pone.0020106
Editor: Christian Gluud, Copenhagen University Hospital, Denmark
Received: October 2, 2010; Accepted: April 25, 2011; Published: May 17, 2011
This is an open-access article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Funding: The authors have no support or funding to report.
Competing interests: The authors have declared that no competing interests exist.
Introduction
Clinical practice guidelines are important for interpreting and translating research evidence for clinical practice and medical decision making. As stated by the Institute of Medicine “guidelines are systematically defined and evidence-based statements that reduce undesirable practice variation by discouraging use of services of questionable value while encouraging use of services with proven efficacy” [1]. However developing reliable, updated and uniform guidelines requires great care and commitment and professional organizations often do not offer detailed recommendations for certain diseases or groups of patients [2]. Moreover, even when guidelines are available, it may be difficult to locate guidelines developed by diverse organizations [3]. For some diseases and questions of interest, sufficient and/or good quality scientific evidence about what to recommend is lacking [4] and subjective recommendations may sometimes be inconsistent across guidelines by different societies, or even harmful [5], [6]. Even when extensive, high-quality evidence exists, there is no assurance that different guidelines make similar recommendations [7] and timely updates are also essential [8], [9], especially when evidence is changing rapidly. When there are guidelines on the same question of interest produced by different organizations, discrepancies in recommendations may generate confusion.
Here we aimed to examine systematically guidelines produced by professional societies and organizations in the fields of cancer and gynecological oncology and evaluate in depth guidelines that address the use of chemotherapy in advanced stage gynecological malignancies, including breast, ovarian, cervical and endometrial cancer. All these malignancies are common [10]. Importantly, they respond differently to chemotherapy, with extensive evidence and clear benefits demonstrated for breast and ovarian cancer and questionable benefits, for advanced endometrial and cervical cancer. Moreover, these malignancies differ a lot in the number of randomized trials that have been conducted on chemotherapy: recent meta-analyses have identified 370, 198, 65, and 13 randomized trials on chemotherapy for advanced stage breast, ovarian, cervical, and endometrial cancer, respectively [11], [12], [13], [14] and the evidence suggests that chemotherapy can be useful for advanced breast and ovarian cancer, but net benefits for hard endpoints (e.g. survival) are limited, if any, for stage III–IVA cervical cancer, and probably trivial or non-existent for stage IVB cervical cancer and advanced endometrial cancer [11], [12], [13], [14]. We performed an extensive search to catalogue relevant societies and organizations worldwide and examined which of them provide any sort of clinical practice guidelines pertinent to chemotherapy for these malignancies. These guidelines were then scrutinized to assess and compare their features and concordance with the available evidence.
Methods
Identification of pertinent societies and organizations
We constructed a database of gynecological and cancer societies and organizations that might provide guidelines for chemotherapy in advanced gynecological malignancies. We considered societies and organizations that were international (with global outlook), intercontinental (including two or more countries in a continent), or national belonging to one of the top 20 countries with the highest human development index [15]. We did not consider private institutions, regional or local societies, regardless of whether they might have produced guidelines or not. Organizations could be either professional societies for specialists or other public not-for-profit organizations.
We performed internet searches in Google and Yahoo (completed in June 2009). Initially we made 624 different searches with each engine and the first 100 results for each search were scrutinized. The 624 searches pertain to all the possible combinations of 8 subject matter terms (“cancer”, “oncology”, “medical oncology”, “radiation oncology”, “surgical oncology”, “cancer research”, “gynecology”, “gynecologic oncology”), three terms for type of entity (“society”, “association”, “organization”), and 26 terms for geographic identifiers (“International”, “European”, “Asian”, “African”, “American”, “Australasian and “Oceanian” and other eligible countries' names). Whenever we came across a URL referring to a potentially pertinent society or organization, we searched the entire directory. All the links in these websites were further searched in order to reach any additional pertinent societies or organizations. We recorded both organizations with accessible webpages, as well as those whose presence was mentioned in some URL, but either their link was not functional (not working or under construction) or they did not have a webpage that we could identify.
Identification of guidelines from pertinent organizations
Available websites from eligible societies and organizations were scrutinized for any sort of clinical practice guidelines on any subject matter (last update June 2009). Whenever there was availability to perform an electronic search within the website, we used the terms “guidelines” or “recommendations” or “position statements” in English. For non-English websites, we translated these terms into the language the website used. We could do this in all languages except for 4 Japanese, 1 Swedish, 1 Finnish, 1 Danish and 2 Dutch organizations.
Additionally, we perused some standard websites containing information and/or links to guideline-related information, including the National Guidelines Clearinghouse (NGC) (www.guideline.gov), Guidelines International Network (G-I-N) (www.g-i-n.net), National Institute for Health and Clinical Excellence (NICE) (www.nice.org.uk), National Library of Health (www.library.nhs.uk), New Zealand Guidelines Group (NZGG) (www.nzgg.org.nz), Guidelines Advisory Committee (GAC) (www.gacguidelines.ca), Australian National Health and Medical Research Council (NHMRC) (www.nhmrc.gov.au) and the Scottish Intercollegiate Guidelines Network (SIGN) (www.sign.ac.uk) in order to identify additional potentially eligible guidelines. Finally, we also searched PubMed with the strategy (guidelines OR recommendations OR position statement) AND (breast OR mammary OR ovarian OR ovary OR endometrial OR endometrium OR uterine OR cervical OR cervix) AND (cancer OR carcinoma OR neoplasm OR neoplasia).
Data extraction – eligible websites
For each pertinent society or organization with an accessible website, we recorded its name, URL, continent and/or country(ies), specialty setting (medical oncology, radiation oncology, surgical oncology, cancer research, or gynecology), and whether it provided any clinical practice guidelines on any topic, regardless of whether the guidelines had been developed by the specific society/organization or some other source and similarly whether they provided any eligible guideline addressing chemotherapy for one or more of the eligible malignancies. Whenever any eligible guidelines were available, we recorded whether recommendations were freely accessible through the website and whether they provided separate information developed by the society/organization itself or a link to another society/organization's guidelines.
Data extraction – eligible guidelines
We recorded the exact wording and documentation regarding the chemotherapy question from all the eligible guidelines that mentioned anything on chemotherapy of advanced or recurrent breast, ovarian, cervical and endometrial cancer. For cervical cancer we separately recorded statements for stages III–IVa and stage IVb since treatment and chemotherapy response differs between these stages [13], [16], [17].
For each eligible guideline we addressed whether it provided grading for the levels of evidence (and how this had been assessed) and the strength of recommendations. We recorded aspects of the production and implementation process for the guideline, including involvement of multidisciplinary teams, search strategies, reported funding, reported conflicts of interest, implementation plans, and use of specific indicators to assess actual uptake. In addition, we recorded the date of publication of the guidelines and the date of publication of the most recent cited randomized trial or meta-analysis. We addressed the utilization of randomized evidence within the guidelines by recording whether guidelines cited within their reference list any randomized trials and meta-analyses pertinent to the chemotherapy question and, if so, how many; and whether they made a comment on the need for more randomized trials to provide sufficient guidance.
We also analyzed the outcomes discussed in the guidelines. We addressed whether the guidelines provided any statement regarding each of the following aspects of the effects of chemotherapy: overall survival, disease-free interval or progression-free survival, response rate, recurrence rate, symptoms relief, quality of life, toxicity profile and cost.
Finally, for each guideline and each type of advanced stage malignancy, we recorded the position on whether chemotherapy should be used or not. We classified guidelines as “in favor” or “against”, if they clearly supported or clearly opposed, respectively, the use of chemotherapy. We also recorded whether guidelines in favor of chemotherapy were unequivocal about its overall favorable benefit-risk ratio or offered any caveats.
Analyses
We evaluated whether guidelines for the different malignancies differed among themselves in the use of grading, aspects of the production and implementation process, timeliness, utilization of randomized evidence, outcomes discussed, and stance towards use of chemotherapy. Group comparisons for categorical variables used Fisher's exact test and group comparisons for continuous variables used Mann-Whitney U test and analysis of variance. All analyses were conducted in STATA SE version 10.0. P-values are two-tailed.
Results
Eligible societies and organizations
Internet searches identified 220 societies/organizations. Additional searches performed in Pubmed and standard guideline websites identified 4 more societies (1 and 3 respectively) that provided any sort of clinical practice guidelines. Of the 224 entities, 105 were named “society”, 51 “association”, 8 “organization” and 60 had other names (alliance, board, centers, coalition, college, consortium, council, federation, forum, foundation, fund, group, institute, league, network, school and union). There were 26 international, 77 intercontinental, and 153 national entities covering a diverse array of countries and a range of specialties (Table 1). Nine of the 224 did not have an accessible webpage, 23 had no functional webpage at all, and 6 had restricted access, thus 192 entities could be accessed and evaluated for the presence of guidelines. Most of them (n = 149) had a webpage in English (Table 1).
Availability of any guidelines
Sixty-nine entities (31%) provided any sort of guidelines in their websites (Table 1). For fifty-nine of them, at least one guideline had been developed by the society/organization itself and in the other 10 there were only links to other societies/organizations' guidelines. Fifty two (75%) made their guidelines available in the English language. The other 17 (25%) provided only guidelines in other languages (4 Italian, 3 German, 3 French, 1 French and German, 2 Dutch, 1 Norwegian, 1 Belgian and 1 Luxemburg). Availability of guidelines did not vary per geographic location and specialty setting (Table 1).
Eligible guidelines for chemotherapy in advanced gynecological malignancies
Twenty entities provided any sort of guideline for chemotherapy in advanced or recurrent gynecological cancer (Table 2). Among them, 14 had developed at least one eligible guideline in English language on their own. This included 8 entities with eligible guidelines for breast cancer chemotherapy [18]–[25], 10 for ovarian cancer [18]–[27], 8 for cervical cancer [18]–[22], [25]–[27], and 8 for endometrial cancer [18]–[22], [26]–[28]. Five had developed guidelines for chemotherapy for all these malignancies [18]–[22].
Grading and timeliness
A little over half of the guidelines covering the use of chemotherapy in advanced gynecological malignancies addressed the level of evidence available for the recommendations; nonetheless no uniformity existed regarding the systems for grading the level of evidence with 5 different systems being utilized (Table 3). Furthermore, less than half provided a grading system for the recommendations with no significant differences across the examined malignancies (Table 3). The timeliness of the guidelines was also comparable among the malignancies tested (Table 3). About two-thirds of the guidelines had been published within the last 5 years and more than half within the last 2 years. No significant difference was observed across guidelines for the different malignancies neither for the date of publication of the most recent cited trial or meta-analysis nor for the number of cited trials or meta-analyses published within the last 5 years.
Key aspects of the production and implementation process
Key aspects regarding the guideline production and implementation process did not significantly differ among analyzed guidelines. More than 60% of the guidelines were stated to have been developed by multidisciplinary teams of experts and only 1 guideline regarding endometrial cancer was developed by experts from one medical field only (obstetricians/gynecologists), while in the remaining guidelines it was unclear what experts exactly were involved. A specific search strategy for the development of the guideline was provided in less than 40%. None of the guidelines reported industry-related funding, but often there was no statement about who had potentially funded the guidelines; moreover, conflicts of interest statements from panel members were available in less than half of the guidelines. Finally, only few of the guidelines provided an implementation plan of the guidelines beforehand or reported that specific indicators were considered in order to assess their actual uptake (Table 3).
Utilization of randomized evidence
The number of randomized trials cited within the guidelines significantly differed among the malignancies tested (p<0.001) (Table 3). Almost all the guidelines for breast and ovarian cancer cited at least one randomized trial within their statements compared to 50% or less of the guidelines for cervical and endometrial cancer. It was also significantly more likely for meta-analyses to be cited in the former than the latter group of malignancies. However, the number of guidelines suggesting the need for more trials to be conducted was comparable for the malignancies tested.
Discussion of outcomes
Statements regarding overall survival were provided in most guidelines with no significant difference for the discussion of this outcome for the different malignancies (Table 4). Conversely, discussion of outcomes such as disease-free survival, response rate, symptoms relief, quality of life, and toxicity of medications significantly differed across malignancies with breast and ovarian cancer guidelines providing more frequently such statements compared to cervical and endometrial cancer. In particular, none of the guidelines on stage III–IVa cervical cancer or endometrial cancer touched on issues of costs related to chemotherapy, whereas less than 40% addressed issues regarding toxicity (Table 4).
Stance towards use of chemotherapy
None of the analyzed guidelines for these advanced stage gynecological malignancies were against using chemotherapy in these patients. All guidelines seemed to endorse explicitly or implicitly the use of some chemotherapy, at least in some settings and patients. No major caveats were raised for breast, ovarian, and stage III–IVa cervical cancer [18]–[27]. For stage IVb cervical cancer three guidelines had some caveats stating that treatment should be individualized (“Patients with stage 4 disease have their treatment very much individualized depending on the distribution of the disease” [27]) or that no standard chemotherapy is available (“No standard chemotherapy treatment is available for patients with stage IVB cervical cancer that provides substantial palliation” [21] and “There is no good single chemotherapy approach that can improve the length of survival in patients with metastatic cervical cancer. Unfortunately, these chemotherapies typically work for only a few months before the cervical cancer begins to grow again. Most patients ultimately succumb to cancer and better treatment strategies are clearly needed” [22]). For endometrial cancer, 3 guidelines only presented data from trials without clear supportive statements that patients would benefit from chemotherapy [20], [21], [27]. All guidelines, with the exception of 1 on endometrial cancer and 1 on cervical cancer, also presented information on and/or endorsement of specific chemotherapeutic agents.
Discussion
Our evaluation identified over 200 cancer and gynecological societies and organizations that operate at international or national level. This is an impressive number and it offers a picture of flourishing professional activity. Many of these organizations have very extensive membership, organize large meetings, and have substantial influence upon their members, subscribers, and visitors to their websites [29]–[32]. Yet only a third of these societies/organizations provided any sort of guidelines within their websites. When it comes to specific questions of interest, such as the use of chemotherapy in advanced stage gynecological malignancies that we analyzed here, only one in ten societies/organizations offered such guidelines, and an even smaller number of them had developed their own guidelines. Therefore, a few professional entities seem to dominate the literature of these influential documents; some others adopt what is produced by others, while the majority is unfamiliar with the guideline concept. Important aspects of the production and implementation process of these guidelines were often unstated. The number of available guidelines did not depend on the bulk of available evidence or the strength of the evidence in favor of chemotherapy. A similar number was available for advanced breast and ovarian cancer where hundreds of trials have been published [11], [12] as well as for advanced endometrial and cervical cancer where the evidence is more limited [13], [14].
The analyzed guidelines did not differ across malignancies in terms of their adoption of levels of evidence, grading of recommendations, and aspects of production and implementation process, or timeliness. Their performance in this regard on average is comparable to guidelines from other fields [33]–[40] and there is substantial room for improvement. The use of levels of evidence and grades of recommendation may be enhanced if this approach is adopted centrally for all guidelines produced by an organization, e.g. the ACC/AHA guidelines in cardiology always use this approach [9]. Timeliness of guidelines updates also constitutes a serious issue [8] and it can affect the reliability of these documents, especially for fields where evidence changes rapidly. There is large variability across medical fields and across clinical questions of interest in the rate with which new evidence accumulates and changes the overall picture [41]–[44]. For all the malignancies analyzed here, most guidelines were updated within the last 5 years. This is probably an acceptable frame for these fields, although it is still possible that a window of 5 years may miss some important recent randomized evidence on specific regimens. One may need to re-assess the acceptable time frame as new evidence arises, but most guidelines developers lack formal procedures for updating their guidelines [45].
The guidelines that we analyzed differed substantially across malignancies in the extent of utilization of randomized evidence (citation of randomized trials and meta-analyses). This is not surprising, given that there are far more randomized trials and meta-analyses performed for advanced breast and ovarian cancer than for cervical and endometrial cancer. The conduct of randomized trials may be encouraged in fields where results of chemotherapy have been favorable (breast and ovarian cancer), while investigators may feel less enthusiastic to perform more trials in fields where chemotherapy has shown equivocal results with no clear benefits (cervical and endometrial cancer). Most of the analyzed guidelines cited at least one randomized trial. It has been observed across diverse fields that utilization of randomized evidence has increased in guidelines over time [46]. Nevertheless, most of the analyzed guidelines cited few randomized trials of those performed on the respective questions. Therefore use of randomized evidence seems to remain eclectic in these documents.
Even though the benefits of chemotherapy vary a lot among the malignancies that we analyzed, all of the guidelines explicitly or implicitly endorsed the use of chemotherapy for all malignancies. This is fully justified for breast, ovarian and possibly stage III–IVa cervical cancer, where the available evidence shows that chemotherapy has substantial benefits [11], [12], [16], [17]. Conversely, chemotherapy has shown no convincing benefits, especially for hard outcomes such as survival, for stage IVb cervical and advanced endometrial cancer [13], [14]. However, only 3 of the 8 guidelines for each of the latter types of tumors had some caveats and were not openly committed to widespread use of chemotherapy in these patients.
Furthermore, outcomes such as secondary effectiveness measures and toxicity profile were addressed more frequently in breast and ovarian cancer, whereas cervical and endometrial cancer guidelines totally omitted statements regarding quality of life and costs of chemotherapy and were less likely to discuss also these other outcomes. Reporting of these issues is important for one to adopt or decline the use of specific treatments. Especially in tumors in which chemotherapy does not appear to offer clear benefit in terms of overall survival, secondary effectiveness outcomes, toxicity, quality of life and cost may be the only data available to justify or not its use. Omission of statements regarding these outcomes in these malignancies may mislead conclusions and may explain the generally favorable stance of the guidelines towards chemotherapy even for advanced cervical and endometrial cancer.
One may argue that scientific societies may sometimes have considerable conflicts that pose obstacles in creating objective guidelines. For example, if professionals benefit from administering specific interventions to patients, their societies may not be unbiased enough to recommend that patients should not be treated with specific interventions (such a recommendation would reduce the volume of clientele and financial gains of these professionals). The ubiquitous recommendation of chemotherapy for all the examined types of cancers (regardless of whether it is effective or not) raises such concerns. An unbiased, intersocietal and interprofessional collaborative approach may be useful in this regard.
Some limitations of our work should be discussed. First, for 32 entities we could not find access to a website. However, it is not very likely that these entities would have guidelines of their own, let alone high-quality ones. Second, it is possible that some entities may not have published guidelines on chemotherapy, if they deemed that chemotherapy is ineffective and thus not worth addressing. However, we found a similar number of guidelines for tumors where chemotherapy is very effective and for those where it is more controversial. Third, there are no established validated searches for unearthing professional societies and organizations and some of them may have been missed by our searches. However, given the multiple layers of our search, it is unlikely that prominent entities were missed.
In conclusion, our evaluation suggests that guidelines from professional gynecological and cancer societies and organizations have substantial room for improvement. Recommendations should be based on solid scientific evidence with balanced discussion of all potential benefits, harms and costs of the proposed interventions.
Author Contributions
Conceived and designed the experiments: JPAI NPP DM. Performed the experiments: JPAI NPP DM. Analyzed the data: JPAI NPP. Contributed reagents/materials/analysis tools: JPAI NPP DM. Wrote the paper: JPAI NPP DM.
References
- 1.
Institute of Medicine (IOM) (2008) “Knowing What Works in Health Care: A Roadmap for the Nation”. Washington DC: The National Academies Press.
- 2. Boyd CM, Darer J, Boult C, Fried LP, Boult L, et al. (2005) Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: implications for pay for performance. JAMA 294: 716–724.
- 3. Somerfield MR, Einhaus K, Hagerty KL, Brouwers MC, Seidenfeld J, et al. (2008) American Society of Clinical Oncology clinical practice guidelines: opportunities and challenges. J Clin Oncol 26: 4022–4026.
- 4. Woolf SH, Grol R, Hutchinson A, Eccles M, Grimshaw J (1999) Clinical guidelines: potential benefits, limitations, and harms of clinical guidelines. BMJ 318: 527–533.
- 5. Woolf SH (1998) Do clinical practice guidelines define good medical care? The need for good science and the disclosure of uncertainty when defining ‘best practices’. Chest 113: 166S–171S.
- 6. Peppercorn J, Burstein H, Miller FG, Winer E, Joffe S (2008) Self-Reported Practices and Attitudes of US Oncologists Regarding Off-Protocol Therapy. J Clin Oncol 26: 5994–6000.
- 7. Eikelboom JW, Guyatt G, Hirsh JW (2008) Guidelines for anticoagulant use in acute coronary syndromes. Lancet 371: 1559–1561.
- 8. Shekelle P, Eccles MP, Grimshaw JM, Woolf SH (2001) When should clinical guidelines be updated? BMJ 323: 155–157.
- 9. Tricoci P, Allen JM, Kramer JM, Califf RM, Smith SC Jr (2009) Scientific evidence underlying the ACC/AHA clinical practice guidelines. JAMA 301: 831–41.
- 10.
Surveillance, Epidemiology and End Results (SEER) Program Available: http://seer.cancer.gov/csr/1975_2006/results_single/sect_01_table.04_2pgs.pdf. Assessed 2010 October 2.
- 11. Mauri D, Polyzos NP, Salanti G, Pavlidis N, Ioannidis JP (2008) Multiple-treatments meta-analysis of chemotherapy and targeted therapies in advanced breast cancer. J Natl Cancer Inst 100: 1780–91.
- 12. Kyrgiou M, Salanti G, Pavlidis N, Paraskevaidis E, Ioannidis JP (2006) Survival benefits with diverse chemotherapy regimens for ovarian cancer: meta-analysis of multiple treatments. J Natl Cancer Inst 98: 1655–63.
- 13. Polyzos NP, Pavlidis N, Paraskevaidis E, Ioannidis JP (2006) Randomized evidence on chemotherapy and hormonal therapy regimens for advanced endometrial cancer: an overview of survival data. Eur J Cancer 42: 319–26.
- 14. Tzioras S, Pavlidis N, Paraskevaidis E, Ioannidis JP (2007) Effects of different chemotherapy regimens on survival for advanced cervical cancer: systematic review and meta-analysis. Cancer Treat Rev 33: 24–38.
- 15.
Human Development ReportsHuman development index 2008. Available: http://hdr.undp.org/en/media/HDR_20072008_Table_1.pdf Assessed 2010 October 2.
- 16. Green JA, Kirwan JM, Tierney JF, et al. (2001) Survival and recurrence after concomitant chemotherapy and radiotherapy for cancer of the uterine cervix: a systematic review and meta-analysis. Lancet 358: 781–786.
- 17. Neoadjuvant chemotherapy for locally advanced cervix cancer. Neoadjuvant Chemotherapy for Cervical Cancer Meta-Analysis Collaboration (NACCCMA) Collaboration (2004) Cochrane Database Syst Rev 2: CD001774.
- 18.
American Cancer Society Available: www.cancer.org Assessed 2009 June 15.
- 19.
European Society of Medical Oncology Available: www.esmo.org Assessed 2009 June 15.
- 20.
National Comprehensive Cancer Network Available: www.nccn.org Assessed 2009 June 15.
- 21.
National Cancer Institute Available: www.cancer.gov Assessed 2009 June 15.
- 22.
National Foundation for Cancer Research Available: www.nfcr.org Assessed 2009 June 15.
- 23.
National Health and Medical Research Council Available: www.nhmrc.gov.au Assessed 2009 June 15.
- 24.
National Institute of Health and Excellence Available: www.nice.org.uk Assessed 2009 June 15.
- 25.
Scottish Intercollegiate Guidelines Network Available: www.sign.ac.uk Assessed 2009 June 15.
- 26.
International Federation of Gynecology and Obstetrics Available: www.figo.org Assessed 2009 June 15.
- 27.
Australian Gynecological Cancer Society Available: www.gcsau.org Assessed 2009 June 15.
- 28.
American College of Obstetricians and Gynecologists Available: www.acog.org Assessed 2009 June 15.
- 29. Rothman DJ, McDonald WJ, Berkowitz CD, Chimonas SC, DeAngelis CD, et al. (2009) Professional medical associations and their relationships with industry: a proposal for controlling conflict of interest. JAMA 301: 1367–72.
- 30. Kesselheim AS, Studdert DM (2007) Role of professional organizations in regulating physician expert witness testimony. JAMA 298: 2907–2909.
- 31. Coyle SL, Ethics and Human Rights CommitteeAmerican College of Physicians-American Society of Internal Medicine (2002) Physician-industry relations. Part 2: organizational issues. Ann Intern Med 136: 403–406.
- 32. Relman AS (2007) Medical professionalism in a commercialized health care market. JAMA 298: 2668–2670.
- 33. Shaneyfelt TM, Mayo-Smith MF, Rothwangl J (1999) Are guidelines following guidelines? The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA 281: 1900–1905.
- 34. Grilli R, Magrini N, Penna A, Mura G, Liberati A (2000) Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 355: 103–106.
- 35. Harpole LH, Kelley MJ, Schreiber G, Toloza EM, Kolimaga J, McCrory DC (2003) Assessment of the scope and quality of clinical practice guidelines in lung cancer. Chest 123: 7S–20S.
- 36. Graham ID, Beardall S, Carter AO, Glennie J, Hébert PC, et al. (2001) What is the quality of drug therapy clinical practice guidelines in Canada? CMAJ 165: 157–163.
- 37. Hasenfeld R, Shekelle PG (2003) Is the methodological quality of guidelines declining in the US? Comparison of the quality of US Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently. Qual Saf Health Care 12: 428–434.
- 38. Arnau JM, Vallano A, Lopez A, Pellisé F, Delgado MJ, et al. (2006) A critical review of guidelines for low back pain treatment. Eur Spine J 15: 543–553.
- 39. Boluyt N, Lincke CR, Offringa M (2005) Quality of evidence-based pediatric guidelines. Pediatrics 115: 1378–1391.
- 40. Nelen WL, van der Pluijm RW, Hermens RP, Bergh C, de Sutter P, et al. (2008) The methodological quality of clinical guidelines of the European Society of Human Reproduction and Embryology (ESHRE). Hum Reprod 23: 1786–1792.
- 41. Patsopoulos NA, Ioannidis JP (2009) The use of older studies in meta-analyses of medical interventions: a survey. Open Medicine 3: 62–68.
- 42. Poynard T, Munteanu M, Ratziu V, Benhamou Y, Di Martino V, et al. (2002) Truth survival in clinical research: an evidence-based requiem? Ann Intern Med 136: 888–895.
- 43. Hall JC, Platell C (1997) Half-life of truth in surgical literature. Lancet 350: 1752.
- 44. Ioannidis JPA (2005) Contradicted and initially stronger effects in highly cited clinical research. JAMA 294: 218–228.
- 45. Shaneyfelt TM, Centor RM (2009) Reassessment of clinical practice guidelines: go gently into that good night. JAMA 301: 868–9.
- 46. Giannakakis IA, Haidich AB, Contopoulos-Ioannidis DG, Papanikolaou GN, Baltogianni MS, et al. (2002) Citation of randomized evidence in support of guidelines of therapeutic and preventive interventions. J Clin Epidemiol 55: 545–55.