Figures
Abstract
Background
Clinical trial results registries may contain relevant unpublished information. Our main aim was to investigate the potential impact of the inclusion of reports from industry results registries on systematic reviews (SRs).
Methods
We identified a sample of 150 eligible SRs in PubMed via backward selection. Eligible SRs investigated randomized controlled trials of drugs and included at least 2 bibliographic databases (original search date: 11/2009). We checked whether results registries of manufacturers and/or industry associations had also been searched. If not, we searched these registries for additional trials not considered in the SRs, as well as for additional data on trials already considered. We reanalysed the primary outcome and harm outcomes reported in the SRs and determined whether results had changed. A “change” was defined as either a new relevant result or a change in the statistical significance of an existing result. We performed a search update in 8/2013 and identified a sample of 20 eligible SRs to determine whether mandatory results registration from 9/2008 onwards in the public trial and results registry ClinicalTrials.gov had led to its inclusion as a standard information source in SRs, and whether the inclusion rate of industry results registries had changed.
Results
133 of the 150 SRs (89%) in the original analysis did not search industry results registries. For 23 (17%) of these SRs we found 25 additional trials and additional data on 31 trials already included in the SRs. This additional information was found for more than twice as many SRs of drugs approved from 2000 as approved beforehand. The inclusion of the additional trials and data yielded changes in existing results or the addition of new results for 6 of the 23 SRs. Of the 20 SRs retrieved in the search update, 8 considered ClinicalTrials.gov or a meta-registry linking to ClinicalTrials.gov, and 1 considered an industry results registry.
Conclusion
The inclusion of industry and public results registries as an information source in SRs is still insufficient and may result in publication and outcome reporting bias. In addition to an essential search in ClinicalTrials.gov, authors of SRs should consider searching industry results registries.
Citation: Potthast R, Vervölgyi V, McGauran N, Kerekes MF, Wieseler B, Kaiser T (2014) Impact of Inclusion of Industry Trial Results Registries as an Information Source for Systematic Reviews. PLoS ONE 9(4): e92067. https://doi.org/10.1371/journal.pone.0092067
Editor: Roberta W. Scherer, Johns Hopkins Bloomberg School of Public Health, United States of America
Received: September 26, 2013; Accepted: February 19, 2014; Published: April 17, 2014
Copyright: © 2014 Potthast et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: This study was funded by the Institute for Quality and Efficiency in Health Care. The funder provided support in the form of salaries for authors (RP, VV, NM, MFK, BW, and TK) and paid the publication fee, but did not have a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.
Competing interests: All authors are employees of the Institute for Quality and Efficiency in Health Care. In order to produce unbiased health technology assessment reports, the institute depends on access to all of the relevant data on the topic under investigation. The authors therefore support the mandatory worldwide establishment of trial registries and study results databases as well as the implementation of reporting standards for clinical trials. This does not alter the authors' adherence to all the PLOS ONE policies on sharing data and materials.
Introduction
The aim of a systematic review (SR) is to identify the complete evidence base of the healthcare intervention under investigation, thus allowing for an unbiased evaluation of the evidence and the formulation of robust recommendations. One step in achieving this aim is to conduct a search in bibliographic databases such as PubMed and EMBASE. But this step alone may be insufficient, as such databases, besides partly containing conference abstracts, generally contain only published information. (In the context of this paper, the term “published” refers to articles published in scientific journals.) However, reporting bias has been shown between different levels of publication (journal publications, conference abstracts, reports from trial registries and results registries, as well as data on file at regulatory agencies and pharmaceutical companies) [1]–[4].
The establishment of trial registries for a priori registration of clinical trials is widely acknowledged as an effective tool to reduce their selective publication [4]–[9]. The first computerized registries were introduced in the United States in the 1960s [5]. Since then, several national and international, public and commercial registries have been created. However, mere knowledge of trial registration is insufficient, as the unbiased assessment of healthcare interventions requires access to the full information on the methods and results of the trials of interest.
On the part of the public sector, in 2007 the Food and Drug Administration (FDA) Amendments Act prescribed mandatory prospective registration, as well as disclosure of specified methods and results, for trials of drugs, biological products or devices regulated by the FDA [10], [11]. For this purpose, the US government's database ClinicalTrials.gov, primarily a trial registry, was expanded and is now the world's largest combined trial and results registry. The trial registry EudraCT was launched by the European Medicines Agency (EMA) in 2004 [12]; however, it is still under construction and the posting of results-related information has only recently become possible [13]. Selected data are publicly accessible via www.clinicaltrialsregister.eu, but no results have yet been posted (status: 13 March 2014).
On the part of the pharmaceutical industry, as early as 2002 the US member companies of the Pharmaceutical Research and Manufacturers Association committed themselves to the registration of all hypothesis-testing clinical trials at initiation and also to the timely disclosure of summary results, regardless of outcome [14], [15], and launched the meta-registry clinicalstudyresults.org in 2004. In 2005 the International Federation of Pharmaceutical Manufacturers and Associations made a similar commitment and launched ifpma.org [16]. Both commitments have since been updated [17], [18]. In addition, several pharmaceutical companies have created their own trial and results registries, albeit not always voluntarily: in the case of GlaxoSmithKline, this was the requirement of a legal settlement after the company had concealed data on antidepressants [19].
However, it is unclear whether the companies unselectively register all trials and disclose all relevant methods and results. Moreover, despite committing themselves to transparency, in numerous cases companies have tried to withhold trial data [14]. Still, industry trial and results registries might represent an important information source for unpublished information on clinical trials.
Previous research has highlighted the potential impact of searching industry results registries: A meta-analysis published in 2007 of trials of the oral antidiabetic rosiglitazone relied heavily on the results registry of the manufacturer GlaxoSmithKline, as 26 of the 42 trials were unpublished. The analysis showed that the drug “was associated with a significant increase in the risk of myocardial infarction and with an increase in the risk of death from cardiovascular causes that had borderline significance”, a finding not reported in published trials [20]. The publication of these data contributed to the market withdrawal of rosiglitazone in many countries in 2010 or to restrictions in prescription [21], [22].
The main aim of our paper was to further investigate the potential impact of the inclusion of reports from industry results registries on SRs. For this purpose, we determined what percentage of a sample of SRs of drugs did not consider industry results registries and whether the inclusion of previously unconsidered reports from these registries led to a change in the results of the SRs.
In the following text the term “registries” refers to trial registries in general; the term “results registries” specifically refers to registries containing trial results. We further distinguish between industry registries (managed by drug manufacturers or industry associations) and public registries (managed by non-profit organizations).
Methods
Selection of systematic reviews
We determined a sample size of 150 SRs on the basis of a pilot study. We identified SRs of drugs in PubMed using a specific systematic review filter (Table S1; search date of the original analysis: 9 November 2009). The potentially relevant citations were listed chronologically according to the most recent entry date. The titles and abstracts of citations were screened by one author. The full texts of potentially relevant citations were obtained and final eligibility was determined by one author and checked by another. Eligible documents were SRs of randomized controlled trials investigating up to 3 individual agents, used alone or in combination therapy. SRs investigating drug classes were not considered. To fulfil the classification of a “systematic” search, the search strategy had to include at least 2 bibliographic databases. If an SR failed to fulfil the above criteria, the next citation listed chronologically in PubMed was assessed until the sample size of 150 SRs was reached (backward selection). The eligible SRs were then screened by one author and checked by another to determine whether results registries of the manufacturers of the drugs under assessment and/or of industry associations had been searched or not.
Search in industry results registries and data extraction
If industry results registries had not been searched, we searched these sources for additional trials not considered in the SR, as well as for additional data on trials already considered. For this purpose, we searched both the results registries of the manufacturers of the drugs under assessment, if available, and of the 2 main industry associations (clinicalstudyresults.org and ifpma.org). The reports retrieved were checked by one author to establish whether the corresponding trials fulfilled the inclusion criteria of the SR. The same author also checked whether the date of the report was within the search period of the SR. If no date was provided, the report was included if the completion date of the trial was at least 2 years before the search date of the SR. The reports classified as relevant were also checked by a second author.
Data analysis
The primary outcomes reported in the SRs were reanalysed, as were the harm outcomes (adverse events and serious adverse events, withdrawals due to adverse events), if available. If no primary outcome was specifically defined in the methods section of the SR, the outcome primarily presented in the results section was used. If feasible, meta-analyses were reconducted according to the methods of the corresponding SR. In the cases where meta-analyses were not feasible, the outcomes presented in the SRs were considered to be “unchanged”.
We then compared the original results of the SRs and the reanalysed results including the data contained in the reports from industry results registries (data on additional trials or additional data on trials already included in the SR). A “change” in a result for a primary or adverse event outcome was defined as either a new relevant result or a change in statistical significance in an existing result (from non-significant to significant or vice versa). The appraisal as to whether results of the SR had changed or not was performed by one author and checked by another.
In all of the above screening, selection and appraisal steps, discrepancies between authors were resolved by consensus.
In an additional post-hoc step, we determined what proportion of SRs investigated newer drugs (i.e. drugs approved from 2000 onwards). We first checked whether the drug was listed as a generic drug in the FDA Orange Book (“Approved drug products with therapeutic equivalence evaluations” [23]). If no information was found in this source, we searched the Internet (including the EMA and FDA websites) to determine the approval status and date of approval.
Search update
In the original analysis of 2009 we did not consider ClinicalTrials.gov, as mandatory posting of results according to the FDA Amendments Act did not apply to trials completed prior to 27 September 2007, and a 12-month time lag for posting applied to trials completed afterwards [10], [24]. The Act would therefore not have covered the vast majority of trials eligible for inclusion in SRs published in 2009. We performed a search update in 2013 to determine whether implementation of the Act had led to the inclusion of ClinicalTrials.gov as a standard information source for SRs and whether the inclusion rate of industry results registries had changed. For this purpose we screened the 20 most recent eligible SRs of drugs available in PubMed before 8 August 2013, which we had selected following the inclusion criteria of the original analysis.
Results
The 150 SRs in the original analysis considered 125 different drugs (Table 1). Half of the SRs investigated neurological and psychiatric, antineoplastic and immunomodulating, or analgesic, antiphlogistic, and antirheumatic agents. 71 were published by the Cochrane Collaboration. 23 of the 125 drugs were newer agents.
133 of the 150 SRs (89%) in the original analysis did not search industry results registries (Figure 1); 12 of the 17 SRs that did were Cochrane reviews. For 23 of the 133 SRs we found 25 additional trials and additional data on 31 trials already included in the SRs. 18 of the 23 SRs contained meta-analyses of at least one outcome (primary outcome or one of the 3 harm outcomes). A reanalysis of the meta-analyses was feasible for at least one outcome in 10 of the 18 SRs.
The inclusion of reports from industry results registries yielded changes in existing results or the addition of new results for 6 SRs (Table 2); 17 SRs remained unchanged. All additional trials and data were identified in results registries of industry associations (in clinicalstudyresults.org for 3 SRs, in impfa.org for 1 SR, and in both for 2 SRs). No further information was identified in results registries of manufacturers. The direction of the effect on the primary outcome or harm outcome reported in the SR was statistically significantly changed in 3 SRs [25]–[27] (Table 3). Two of these changes were to the disadvantage of the test drug for an SR of valproate in schizophrenia (adverse event rate) [27] and for an SR of sibutramine in obesity (blood pressure values) [26]. One change was to the advantage of the test drug for an SR of olanzapine in bipolar disorder (withdrawal rate due to adverse events) [25].
For 2 SRs of antidepressants [28], [29] we identified additional trials or additional data for comparisons not reported in the SR. An SR of milnacipran [28] contained no data on milnacipran versus venlafaxine: we identified a trial that reported both efficacy and safety data for this comparison. For an SR on sertraline [29] we identified an additional trial presenting previously unreported treatment-emergent adverse events for the comparison of sertraline versus reboxetine; this trial also provided additional data on withdrawals due to adverse events. Finally, we identified additional data for a comparison of adverse event rates reported in an SR on ezetimibe in hypercholesterolaemia [30]. In all of the 4 comparisons above the differences between treatment groups were not statistically significant.
With regard to the provision of additional information in industry results registries, there was a notable difference between newer and older drugs: we found additional trials or data for 35% of SRs of drugs approved from 2000 onwards (8 out of 23 drugs) but for only 16% of SRs of drugs approved beforehand (15 out of 96 drugs) (Table S2).
Nine of the 20 SRs (45%) in our search update of August 2013 considered results registries, of which 8 searched ClinicalTrials.gov or a meta-registry linking to this source, and 1 searched an industry results registry. Seven of the 20 SRs were Cochrane reviews of which 6 considered results registries (Table S3).
Discussion
Summary of findings
Our original analysis showed that few SRs consider industry results registries and that results of some SRs change if reports from these registries are considered. The identified additional relevant trials and data on trials already included in the SRs mainly concerned drugs approved from 2000 onwards, which is not surprising, as industry results registries were not introduced until later. A search in such registries may thus be particularly worthwhile for SRs of newer drugs. Our search update showed that less than half of the SRs considered ClinicalTrials.gov as an information source, and that industry results registries were hardly considered.
Previous research on industry and public results registries
Both our study and the meta-analysis of rosiglitazone trials already cited [20] refer to the impact of industry results registries; similar findings have been shown for public sources. As early as 1986, Simes compared results of published cancer trials and trials identified in the International Research Cancer Databank [4]. Whilst meta-analyses of published trials on ovarian cancer and multiple myeloma showed a significant survival advantage of combination chemotherapy versus treatment containing an initial alkylating agent, this advantage was absent or at least reduced in the meta-analyses of registered trials.
Two studies conducted by our Institute further highlight the relevance of information sources containing unpublished data. In 2012 we published a comparison of the completeness of reporting of information sources used for 268 trials included in 16 SRs of drugs prepared between 2006 and February 2011 [31]. Three document types were examined: journal publications, reports from industry results registries, and clinical study reports previously on file at pharmaceutical companies. Compared with journal publications, reporting quality was poorer in reports from industry results registries for methods items (P<0.001), but better for outcomes (primary outcomes and adverse events; P = 0.005); however, both sources were clearly inferior to clinical study reports. A second recently published analysis of a subsample of the above trials also showed major discrepancies between all 3 information sources regarding the completeness of reporting of several (both primary and secondary) patient-relevant outcomes such as clinical events and symptoms, quality of life, and adverse events [32]. Neither study investigated the impact of unpublished data on the actual numerical results reported in publications: however, the substantial differences in completeness of information highlight the risk of bias in published results.
Deficits of trial and results registration
Trial registries may also provide valuable information for planned SRs or updates, as they list ongoing trials, including those soon to be completed. However, even if all SRs used registries as an information source, this would only be an approximation to the complete evidence on a topic and would thus merely reduce, but not eliminate the problem of publication bias. Firstly, as long as there is no worldwide mandatory registration of trials and trial results, the pool of trials in registries may also be incomplete, as trials may still be “hidden” on file in pharmaceutical companies and regulatory agencies. Secondly, even if a trial is registered, the information provided may be inadequate: a retrospective evaluation of 21 industry and public trial registries showed that, although compliance with the registration criteria of the World Health Organization had improved between the years 2005 and 2007, individual registry entries on study characteristics and methods were largely incomplete [33].
Despite the FDA Amendments Act of 2007, registration of results is also incomplete: a recently published analysis of 585 large randomized trials registered in ClinicalTrials.gov and completed prior to January 2009 showed that 171 (29%) remained unpublished more than 3 years later; 133 (78%) of the unpublished trials had no results available in ClinicalTrials.gov [34]. Furthermore, the Act contains a major loophole, as it was prospective and thus does not cover trials of many drugs widely prescribed in clinical practice [35]. The format originally proposed for industry results registries was based on the synopsis of a clinical study report according to ICH E3 [18], [19]. However, the ICH E3 synopsis was developed to accompany a full clinical study report and is an insufficient representation of a clinical trial. For instance, a recently published comparison of internal company synopses and journal publications on trials of the oral antidiabetic repaglinide showed that, in addition to other inconsistencies, the reporting of deaths was incomplete in both types of documents [36].
The well-known problem of selective outcome reporting in publications [37]–[40] seems to be independent of whether a trial is registered or not [41]. The registration of trial protocols (including any amendments) and of full results can help to identify this type of bias [40], [41] and is also called for in the Ottawa statement [6].
It should also be noted that industry registries may possibly contain more biased information than public ones, as biased reporting of industry-sponsored trials and meta-analyses is a well-known problem [42]–[44]. However, despite the noted potential deficits of trial registries, as long as the effort required is reasonable, SRs should at least use all available information sources, i.e. also include industry and public registries. This is all the more important as registered trials are often not published in scientific journals. This has been shown both for industry (see the rosiglitazone example above) and public registries: less than half (311 of 677, 46%) of a subsample of trials registered in ClinicalTrials.gov and completed prior to January 2006 were published 2 years after completion [45].
Limitations
We only investigated a relatively small range of drugs and a limited number of outcomes. In addition, we did not screen the results registries of manufacturers of comparator drugs to check whether they had also conducted trials investigating the test drugs; these trials may not have been reported elsewhere. These limitations could potentially lead to an underestimation of the effect. It should also be noted that a change in the results of the SRs was defined as either the addition of new results or a change in the statistical significance of an existing result based on p-values. Although statistical significance is the usual criterion to determine whether an intervention shows an advantage or disadvantage against a comparator, statistically significant changes may not necessarily be clinically relevant and alter the conclusions of an SR.
One could argue that our study is outdated. As stated, in our original analysis we did not consider Clinicaltrials.gov, as mandatory results registration did not apply to the vast majority of trials eligible for inclusion in the SRs analysed. For instance, 49 of the 56 additional trials and data sets we identified in industry results registries had been completed before mandatory results registration became effective. As the number of industry-sponsored and publicly-funded registered trials and trial results is growing, ClinicalTrials.gov is becoming an increasingly important information source. One would thus assume that in general more recent SRs would consider this source. However, as our search update showed, only 40% of the 20 eligible SRs available in PubMed in August 2013 searched ClinicalTrials.gov or a meta-registry linking to this source, and only 5% searched an industry results registry.
With the increasing relevance of ClinicalTrials.gov, that of industry meta-registries is decreasing: clinicalstudyresults.org went offline in 2012; the expansion of ClinicalTrials.gov was cited as a reason for this measure [46]. Some reports on trials previously available on clinicalstudyresults.org are now available in manufacturer registries (see reference list), indicating that searching manufacturer registries may now be more worthwhile. The industry meta-registry impfa.org, where we identified additional trials and data for 3 SRs, is still online and thus also a potentially relevant information source.
Further developments
In addition to the FDA Amendments Act, a further major development regarding the availability of clinical trial data is the plan by EMA to release full clinical trial data from 2014 onwards for all newly approved drugs [47], [48]. However, industry has taken legal action to prevent EMA releasing data under its current policy [49] and it is unclear whether EMA will be able to fully implement the new policy.
It is not surprising that the majority of SRs considering industry results registries in our study were Cochrane reviews. Cochrane reviews are generally of better reporting quality and typically search substantially more databases than non- Cochrane reviews [50]; in addition, the Cochrane Handbook acknowledges the increasing importance of searching trial registries [51]. However, less than 40% of Cochrane reviews have been shown to search trial registries [52] and a recent survey of over 2000 Cochrane authors reported that only about 6% and 1% obtained unpublished data from public and industry registries respectively [53]. Another guideline, the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) Statement, also recommends searching trial registries [54]. If these guidelines defined such a search as a mandatory component of any systematic search for evidence, this could provide an effective means to increase the inclusion rate of registries as an information source for SRs. For this purpose, it would be helpful if all public and industry registries were listed on a central website, to ensure complete coverage of all these sources.
Conclusions
The inclusion of industry and public results registries as an information source in SRs is still insufficient and may result in publication and outcome reporting bias. In addition to an essential search in ClinicalTrials.gov, authors of SRs should consider searching industry results registries.
Supporting Information
Table S1.
Search strategy for identifying systematic reviews of drugs in Medline (OVID).
https://doi.org/10.1371/journal.pone.0092067.s001
(DOC)
Table S2.
Number of drugs for which additional trial(s) or data were found in industry results registries, according to year of drug approval.
https://doi.org/10.1371/journal.pone.0092067.s002
(DOC)
Table S3.
Inclusion rate of results registries in systematic reviews of drugs identified in PubMed in August 2013 (search update).
https://doi.org/10.1371/journal.pone.0092067.s003
(DOC)
Acknowledgments
The authors thank Elke Hausner and Verena Wekemann for supporting information retrieval.
Author Contributions
Conceived and designed the experiments: TK RP. Performed the experiments: RP VV MFK BW TK. Analyzed the data: VV. Wrote the paper: RP NM. Revised the manuscript draft and approved the final version: RP NM VV MFK BW TK.
References
- 1. Hopewell S, McDonald S, Clarke M, Egger M (2007) Grey literature in meta-analyses of randomized trials of health care interventions. Cochrane Database Syst Rev MR000010.
- 2. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.
- 3. Eyding D, Lelgemann M, Grouven U, Harter M, Kromp M, et al. (2010) Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials. BMJ 341: c4737.
- 4. Simes RJ (1986) Publication bias: the case for an international registry of clinical trials. J Clin Oncol 4: 1529–1541.
- 5. Dickersin K, Rennie D (2003) Registering clinical trials. JAMA 290: 516–523.
- 6. Krleza-Jeric K, Chan AW, Dickersin K, Sim I, Grimshaw J, et al. (2005) Principles for international registration of protocol information and results from human trials of health related interventions: Ottawa statement (part 1). BMJ 330: 956–958.
- 7. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, et al. (2004) Clinical trial registration: a statement from the International Committee of Medical Journal Editors. N Engl J Med 351: 1250–1251.
- 8. Laine C, Horton R, DeAngelis CD, Drazen JM, Frizelle FA, et al. (2007) Clinical trial registration. BMJ 334: 1177–1178.
- 9. Sim I, Chan AW, Gulmezoglu AM, Evans T, Pang T (2006) Clinical trial registration: transparency is the watchword. Lancet 367: 1631–1633.
- 10.
United States Congress (2007) Food and Drug Administration Amendments Act (FDAAA) of 2007: public law no. 110–85. Available: http://www.gpo.gov/fdsys/pkg/PLAW-110publ85/pdf/PLAW-110publ85.pdf. Accessed 4 September 2013.
- 11. Groves T (2008) Mandatory disclosure of trial results for drugs and devices. BMJ 336: 170.
- 12.
European Agency for the Evaluation of Medicinal Products (2004) EMEA launches EudraCT database. Available: http://www.ema.europa.eu/docs/en_GB/document_library/Press_release/2009/11/WC500015292.pdf. Accessed 4 September 2013.
- 13.
European Medicines Agency (2013) EudraCT: European Union Drug Regulating Authorities Clinical Trials [Website]. Available: https://eudract.ema.europa.eu/. Accessed 6 Aug 2013.
- 14. McGauran N, Wieseler B, Kreis J, Schüler YB, Kölsch H, et al. (2010) Reporting bias in medical research: a narrative review. Trials 11: 37.
- 15.
Pharmaceutical Research and Manufacturers of America (2002) Principles on conduct of clinical trials and communication of clinical trial results.
- 16.
International Federation of Pharmaceutical Manufacturers and Associations (2005) Joint position on the disclosure of clinical trial information via clinical trial registries and databases.
- 17.
Pharmaceutical Research and Manufacturers of America (2011) Principles on the conduct of clinical trials; communication of clinical trial results. Available: http://phrma.org/sites/default/files/pdf/042009_clinical_trial_principles_final_0.pdf. Accessed 5 September 2013.
- 18.
International Federation of Pharmaceutical Manufacturers and Associations (2009) Joint position on the disclosure of clinical trial information via clinical trial registries and databases. Available: http://clinicaltrials.ifpma.org/clinicaltrials/fileadmin/files/pdfs/EN/November_10_2009_Updated_Joint_Position_on_the_Disclosure_of_Clinical_Trial_Information_via_Clinical_Trial_Registries_and_Databases.pdf. Accessed 4 September 2013.
- 19.
Office of the Attorney General (2004) Settlement sets new standard for release of drug information. Available: http://www.ag.ny.gov/press-release/settlement-sets-new-standard-release-drug-information. Accessed 19 September 2013.
- 20. Nissen SE, Wolski K (2007) Effect of rosiglitazone on the risk of myocardial infarction and death from cardiovascular causes. N Engl J Med 356: 2457–2471.
- 21.
European Medicines Agency (2010) European Medicines Agency recommends suspension of Avandia, Avandamet and Avaglim: anti-diabetes medication to be taken off the market. Available: http://www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/09/news_detail_001119.jsp&mid=WC0b01ac058004d5c1. Accessed 9 December 2013.
- 22.
Food and Drug Administration (2010) FDA significantly restricts access to the diabetes drug Avandia. Available: http://www.fda.gov/Drugs/DrugSafety/PostmarketDrugSafetyInformationforPatientsandProviders/ucm226956.htm. Accessed 9 December 2013.
- 23.
Food and Drug Administration, editor (2011) Approved drug products with therapeutic equivalence evaluations: 31st edition. Rockville: FDA.
- 24. Wood AJJ (2009) Progress and deficiencies in the registration of clinical trials. N Engl J Med 360: 824–830.
- 25. Cipriani A, Rendell JM, Geddes J (2009) Olanzapine in long-term treatment for bipolar disorder. Cochrane Database Syst Rev CD004367.
- 26. De Lemos HP Jr, Atallah AN, Alves De Lemos AL (2008) Can sibutramine alter systemic blood pressure in obese patients? Systematic review and meta-analysis. Sao Paulo Med J 126: 342–346.
- 27. Schwarz C, Volz A, Li C, Leucht S (2008) Valproate for schizophrenia. Cochrane Database Syst Rev CD004028.
- 28. Nakagawa A, Watanabe N, Omori IM, Barbui C, Cipriani A, et al. (2009) Milnacipran versus other antidepressive agents for depression. Cochrane Database Syst Rev CD006529.
- 29. Cipriani A, La Ferla T, Furukawa TA, Signoretti A, Nakagawa A, et al. (2009) Sertraline versus other antidepressive agents for depression. Cochrane Database Syst Rev CD006117.
- 30. Ara R, Tumur I, Pandor A, Duenas A, Williams R, et al. (2008) Ezetimibe for the treatment of hypercholesterolaemia: a systematic review and economic evaluation. Health Technol Assess 12: iii, xi–212, iii, xi-xiii, 1-212.
- 31. Wieseler B, Kerekes MF, Vervoelgyi V, McGauran N, Kaiser T (2012) Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications. BMJ 344: d8141.
- 32. Wieseler B, Wolfram N, McGauran N, Kerekes MF, Vervolgyi V, et al. (2013) Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data. PLoS Med 10: e1001526.
- 33. Moja LP, Moschetti I, Nurbhai M, Compagnoni A, Liberati A, et al. (2009) Compliance of clinical trial registries with the World Health Organization minimum data set: a survey. Trials 10: 56.
- 34. Jones CW, Handler L, Crowell KE, Keil LG, Weaver MA, et al. (2013) Non-publication of large randomized clinical trials: cross sectional analysis. BMJ 347: f6104.
- 35. Turner EH, Moaleji NJ, Arnold BL (2008) Closing a loophole in the FDA Amendments Act. Science 322: 44–46.
- 36. Schroll J (2013) Deaths in trials should always be reported. BMJ 347: f4219.
- 37. Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan AW, et al. (2008) Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 3: e3081 Available: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0003081. Accessed 4 September 2013.
- 38. Chan AW, Altman DG (2005) Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 330: 753.
- 39. Melander H, Ahlqvist-Rastad J, Meijer G, Beermann B (2003) Evidence b(i)ased medicine: selective reporting from studies sponsored by pharmaceutical industry; review of studies in new drug applications. BMJ 326: 1171–1173.
- 40. Vedula SS, Bero L, Scherer RW, Dickersin K (2009) Outcome reporting in industry-sponsored trials of gabapentin for off-label use. N Engl J Med 361: 1963–1971.
- 41. Rasmussen N, Lee K, Bero L (2009) Association of trial registration with the results and conclusions of published trials of new oncology drugs. Trials 10: 116.
- 42. Jorgensen AW, Hilden J, Gotzsche PC (2006) Cochrane reviews compared with industry supported meta-analyses and other meta-analyses of the same drugs: systematic review. BMJ 333: 782.
- 43. Lexchin J, Bero LA, Djulbegovic B, Clark O (2003) Pharmaceutical industry sponsorship and research outcome and quality: systematic review. BMJ 326: 1167–1170.
- 44. Bekelman JE, Li Y, Gross CP (2003) Scope and impact of financial conflicts of interest in biomedical research: a systematic review. JAMA 289: 454–465.
- 45. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009) Trial publication after registration in clinicaltrials.gov: a cross-sectional analysis. PLoS Med 6: e1000144 Available: http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000144. Accessed 4 September 2013.
- 46.
Veitch E (2012) Silent takedown of the pharma trials database…and more. In: PLOS Blogs. Available: http://blogs.plos.org/speakingofmedicine/2012/03/23/silent-takedown-of-the-pharma-trials-database%E2%80%A6and-more/. Accessed 8 August 2013.
- 47.
European Medicines Agency (2010) European Medicines Agency widens public access to documents: policy on access to documents also sets out new approach for proactive disclosure of documents. Available: www.ema.europa.eu/ema/index.jsp?curl=pages/news_and_events/news/2010/11/news_detail_001158.jsp&mid=WC0b01ac058004d5c1&murl=menus/news_and_events/news_and_events.jsp. Accessed 9 December 2013.
- 48.
European Medicines Agency (2013) Draft policy 70: publication and access to clinical-trial data. Available: http://www.emea.europa.eu/ema/index.jsp?curl=pages/includes/document/document_detail.jsp?webContentId=WC500144730&mid=WC0b01ac058009a3dc. Accessed 1 July 2013.
- 49. Jack A (2013) Pharma group sues European regulator over data. Financial Times Available: http://www.ft.com/cms/s/0/9956b5ca-899f-11e2-92a0-00144feabdc0.html#axzz2dug4fbrM. Accessed 4 September 2013.
- 50. Moher D, Tetzlaff J, Tricco AC, Sampson M, Altman DG (2007) Epidemiology and reporting characteristics of systematic reviews. PLoS Med 4: e78 Available: http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.0040078. Accessed 4 September 2013.
- 51.
Lefebvre C, Manheimer E, Glanville J (2008) Searching for studies. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. New York: Wiley. pp. 95–150.
- 52. van Enst WA, Scholten RJ, Hooft L (2012) Identification of additional trials in prospective trial registers for Cochrane systematic reviews. PLoS One 7: e42812 Available: http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0042812. Accessed 4 September 2013.
- 53. Schroll JB, Bero L, Gotzsche PC (2013) Searching for unpublished data for Cochrane reviews: cross sectional study. BMJ 346: f2231.
- 54. Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, et al. (2009) The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: explanation and elaboration. PLoS Med 6: e1000100 Available: http://www.plosmedicine.org/article/info%3Adoi%2F10.1371%2Fjournal.pmed.1000100. Accessed 9 December 2013.
- 55.
Abbott Laboratories A randomized, double-blind study of the safety and efficacy of Depakote ER plus an atypical antipsychotic vs. an atypical antipsychotic alone in the treatment of schizophrenia: clinical study report M02-547. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_782_0.pdf. Accessed 4 October 2009 (no longer available). Now available: http://www.abbott.com/static/cms_workspace/content/document/Citizenship/clinical_study_results/divalproex-sodium_M02-547-study.pdf. Accessed 25 November 2013.
- 56.
Abbott Laboratories (2006) Blood pressure and weight trajectory on a dual antihypertensive combination plus sibutramine versus placebo in obese hypertensives: clinical study report KD 200010. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_1275_0.pdf. Accessed 24 February 2010 (no longer available). Now available: http://www.abbott.com/static/cms_workspace/content/document/Citizenship/clinical_study_results/sibutramine_kd200010-study.pdf Accessed 25 November 2013.
- 57.
Abbott Laboratories (2006) Randomized trial of obese non-diabetic Malaysians using sibutramine: a randomized double-blind placebo-controlled study of sibutramine in the management of obese subjects in Malaysia; clinical trial report MLAY02001. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_4130_0.pdf. Accessed 24 February 2010 (no longer available). Now available: http://www.abbott.com/static/cms_workspace/content/document/Citizenship/clinical_study_results/sibutramine_mlay-02-001-study.pdf. Accessed 25 November 2013.
- 58.
Abbott Laboratories A multicenter, double-blind, randomized, placebo-controlled, parallel group study to examine the longer-term efficacy and safety of Meridia (sibutramine hydrochloride) in obese subjects; clinical study report SB118. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_1146_0.pdf. Accessed 24 February 2010 (no longer available). Now available: http://www.abbott.com/static/cms_workspace/content/document/Citizenship/clinical_study_results/sibutramine_sb118-study.pdf. Accessed 25 November 2013.
- 59. Berkowitz RI, Fujioka K, Daniels SR, Hoppin AG, Owen S, et al. (2006) Effects of sibutramine treatment in obese adolescents: a randomized trial. Ann Intern Med 145: 81–90.
- 60.
Eli Lilly (2004) Olanzapine added to mood stabilizer for relapse prevention in patients with bipolar disorder: CT registry ID#1035b. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_173_0.pdf. Accessed 13 April 2009 (no longer available). Now available: http://www.lillytrials.com/results/Zyprexa.pdf. Accessed 25 November 2013.
- 61.
Eli Lilly (2004) Olanzapine versus placebo in the prevention of relapse in bipolar disorder: clinical study summary; study F1D-MC-HGHL. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_1164_0.pdf Accessed 13 April 2009 (no longer available). Now available: http://www.lillytrials.com/results/Zyprexa.pdf. Accessed 25 November 2013.
- 62.
Eli Lilly (2004) Olanzapine versus divalproex in the treatment of acute mania: clinical study summary; study F1D-US-HGHQ. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_176_0.pdf. Accessed 13 April 2009 (no longer available). Now available: http://www.lillytrials.com/results/Zyprexa.pdf. Accessed 25 November 2013.
- 63.
Eli Lilly (2004) Olanzapine versus lithium in relapse prevention in bipolar disorder: clinical study summary; study F1D-MC-HGHT. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_603_0.pdf. Accessed 13 April 2009 (no longer available). Now available: http://www.lillytrials.com/results/Zyprexa.pdf. Accessed 25 November 2013.
- 64.
Wyeth (2006) A multicenter, double-blind comparative study to administer venlafaxine ER (18.75 mg/day), venlafaxine ER (75 to 150 mg/day) or milnacipran hydrochloride (50 to 100 mg/day) for 12 weeks to patients with major depressive disorder to demonstrate non-inferiority of venlafaxine ER (75 to 150 mg/day) to milnacipran hydrochloride in antidepressive effect based on the HAM-D17 response rate: protocol 0600B1-816. In: ClinicalStudyResults.org. Available: http://www.clinicalstudyresults.org/documents/company-study_9750_1.pdf. Accessed 20 February 2010 (no longer available).
- 65.
Pfizer (2008) Sertraline/[S,S]-reboxetine combination versus sertraline and [S,S]-reboxetine monotherapy in major depressive disorder (MDD) in a double-blind, placebo-controlled, eight week study: protocol no. A0501075. In: ClinicalStudyResults.org. Available: www.clinicalstudyresults.org. Accessed 20 February 2010 (no longer available).
- 66.
Pfizer (2008) A multicenter randomized, double-blind, parallel-group study of sertraline versus venlafaxine XR in the acute treatment of outpatients with major depressive disorder. In: Clinical Study Results Database. Available: www.clinicalstudyresults.org. Accessed 20 February 2010 (no longer available).
- 67.
Glaxo Smith Kline (2005) A multicenter, double-blind, placebo-controlled comparison of the effects on sexual functioning of wellbutrin (bupropion HCI) sustained release and sertraline in outpatients with moderate to severe recurrent major depression (WELL AK1A4001). In: Clinical Study Results Database. Available: www.clinicalstudyresults.org Accessed 20 February 2010 (no longer available).
- 68.
Glaxo Smith Kline (2005) A multicenter, double-blind, placebo-controlled comparison of the effects on sexual functioning of wellbutrin (bupropion HCl) sustained release and sertraline in outpatients with moderate to severe recurrent major depression (WELL AK1A4002). In: Clinical Study Results Database. Available: www.clinicalstudyresults.org Accessed 20 February 2010 (no longer available).
- 69.
Glaxo Smith Kline (2005) The safety and efficacy of wellbutrin sustained release and sertraline in depressed outpatients. In: Clinical Study Results Database. Available: www.clinicalstudyresults.org Accessed 20 February 2010 (no longer available).
- 70.
Merck (2005) A multicenter, double-blind, randomized study to evaluate the lipid-altering efficacy, safety, and tolerability of ezetimibe coadministered with simvastatin versus simvastatin monotherapy in African-American subjects with primary hypercholesterolemia (protocol no. P03377). In: ClinicalStudyResults.org. Available: www.clinicalstudyresults.org. Accessed 10 July 2011 (no longer available).