Advertisement
Research Article

Result Publication of Chinese Trials in World Health Organization Primary Registries

  • Liu Xuemei,

    Affiliation: Chinese Journal of Evidence-Based Medicine, West China Periodicals Press, West China Hospital of Sichuan University, Chengdu, China

    X
  • Li Youping mail,

    yzmylab@hotmail.com

    Affiliation: Chinese Evidence-Based Medicine Center, West China Hospital of Sichuan University, Chengdu, China

    X
  • Yin Senlin,

    Affiliation: West China School of Medicine of Sichuan University, Chengdu, China

    X
  • Song Shangqi

    Affiliation: West China School of Medicine of Sichuan University, Chengdu, China

    X
  • Published: September 13, 2010
  • DOI: 10.1371/journal.pone.0012676

Abstract

Background

Result publication is the key step to improve the transparency of clinical trials.

Objective

To investigate the result publication rate of Chinese trials registered in World Health Organization (WHO) primary registries.

Method

We searched 11 WHO primary registries for Chinese trials records. The progress of each trial was analyzed. We searched for the full texts of result publications cited in the registration records. For completed trials without citations, we searched PubMed, Embase, Chinese Biomedical Literature Database (Chinese), China Knowledge Resource Integrated Database, and Chinese Science and Technology Periodicals Database for result publications. The search was conducted on July 14, 2009. We also called the investigators of completed trials to ask about results publication.

Results

We identified 1294 Chinese trials records (428 in ChiCTR,743 in clinicaltrials.gov,55 in ISRCTN, 21 in ACTRN). A total of 443 trials had been completed. The publication rate of the Chinese trials in WHO primary registries is 35.2%(156/443).The publication rate of Chinese trials in clinicaltrials.gov, ChiCTR, ISRCTN, and ACRTN was 36.5% (53/145), 36.3% (89/245), 26.0%(9/44), and 55.6%(5/9), respectively. The publication rate of trials sponsored by industry(23.8%) was lower than that of sponsored by central and local government(31.7%), hospital(35.1%), and universities (40.7%). The publication rate for randomized trials was higher than that of cohort study and case-control study (33.2% versus 16.7%, 22.2%). The publication rate for interventional studies and observational studies was similar(33.4% versus 33.3%).

Conclusion

The publication rate of the registered Chinese trials was low, with no significant difference between ChiCTR and clinicaltrials.gov. An effective mechanism is needed to promote publication of results for registered trials in China.

Introduction

Trial results published in peer-reviewed journals are a major source of scientific information for health policymakers, patients, doctors, and researchers. However, non-publication of results and selective outcomes reporting make the available body of evidence incomplete and possibly biased. This can lead to duplicate research and funding, and to harm for patients whose health care was not informed by the best evidence [1][5]. Positive results are usually considered more interesting (and publishable) than negative results, and the results of many trials are under disclosure. Therefore, published trials together provide an incomplete picture of overall research results [6][22]. Clinical trial registration was thus established to improve the transparency of clinical trials, reducing selective outcome reporting.

In May 2007, the International Clinical Trials Registry Platform of the World Health Organization (WHO ICTRP) was established to offer an international standard for trial registration. ICTRP made trial registration the rule, not the exception. As of July 2009,10 WHO primary registries in Australia and New Zealand, the United Kingdom, China, India, Germany, Japan, the Netherlands, Iran, Sri Lanka, and Africa are approved by WHO ICTRP. Another registry, Clinicaltrials.gov, is approved by the International Committee of Medical Journal Editors (ICMJE)[10]. ICMJE accepts registration of trials in any of the WHO ICTRP primary registries, and WHO ICTRP accepts registrations approved by ICMJE [11][14]. FDAAA 801(section 801 of FDAAA, a United States public law covering clinical trial) also calls on the US National Institutes of Health to require that Clinicaltrials.gov entries be augmented with trial results within 12 months of trial completion or within 30 days of FDA approval (or clearance) of a new drug, biologic, or device (with the exception of phase I clinical trials and complementary and alternative medicine trials)[13], [15]. Our previous study showed that Chinese trials are registered not only in the Chinese Clinical Trial Register (ChiCTR), but also in other WHO ICTPR primary registries or ICMJE-approved registries[16].

Many trials are conducted in China. The State Food and Drug Administration (SFDA) of China receives about 1,250 applications for authorization of new trials and new drug applications each month [17]. Up to now, no published studies have explored the result publication rate in China after registration. We aimed to assess the result publication rate in the 10 WHO registries and to identify factors associated with publication.

Materials and Methods

Inclusion and exclusion criteria

We included trials we considered to be “sponsored by China”—that is, either the principal investigator, the main funder, or both were located in China. We excluded trials in which study participants were Chinese, but the trial was not sponsored by China.In this study, a Chinese trial could be either interventional or observational.

Data collection

We searched 11 registries for trials sponsored by China: the Australian and New Zealand Clinical Trials Registry (ACTR), the International Standard Randomised Controlled Trial Number Register (ISRCTN), the Chinese Clinical Trial Registry (ChiCTR), the Iranian Registry of Clinical Trials (IRCT), the German Clinical Trials Register (DRKS), the Japan Primary Registries Network, the Netherlands National Trial Register (NTR), the Sri Lanka Clinical Trials Registry (SLCTR), the Clinical Trials Registry – India (CTRI), the Pan African Clinical Trial Registry (PACTR), and Clinicaltrials.gov.

All records in ChiCTR were included in our analysis, and we used “China” as a key word in searching the other 10 registries for potentially eligible trials. The search was conducted on July 14, 2009.

Data extraction

We designed a program using the programming language Python to extract information about clinical trials. The program first automatically fetches web pages of clinical trials from registration centers and stores them in separate HTML files. Then it utilizes specific tags and key words in HTML files to locate each of the registration items, and extracts them. Finally, this program compiles the extracted data and exports it into an Excel table for further analysis. The first search was conducted on December 31, 2009 and included 920 trial records. To guarantee the reliability and reproducibility of the program, we checked the 920 records manually. Our check confirmed that the program could extract the data correctly. The rate of agreement is 100%. The final search was conducted on July 14, 2009.

The progress status of a trial was indexed as completed, ongoing, terminated, or suspended. To find the full text of each completed trial, we proceeded as follows (see figure 1)

  1. We looked for links to published results in the registry.
  2. If there were no links to published results in the registry, we used key words from the record to search systematically for the published study in PubMed (English language, 1966 to July 14, 2009), Embase (English language, 1974 to July 14, 2009), CNKI (Chinese language, 1979 to July 14, 2009), Wanfang database (Chinese language, 1990 to July 14, 2009), and CBM (Chinese language, 1978 to July 14, 2009). We identified a citation as the result publication by matching registration number, the authors, institute, interventions, participants, outcome measures, study design, and study start date in the registration record.
  3. If contact information was available in the registry, we called or emailed the investigators to ask for results publication information.
thumbnail

Figure 1. Means of obtaining full text of completed trials.

doi:10.1371/journal.pone.0012676.g001

Two authors (Liu XM and Yin SL) devised the search strategy. Two authors (Yin SL and Song SQ) independently searched the databases, and shared the work of calling or emailing the authors. Differences were resolved by discussion among all the authors.

Publication of trial result

The publication rate (the proportion of included trials with result publications) was calculated. To examine the associations between trial characteristics and publication rate, we calculated the publication rate for trials at different stages, and with various sponsor types, study designs, and intervention types. The trial stage was categorized as completed, ongoing, terminated, or suspended; sponsor types were industry, central or local government body, hospital, or university; study designs were observational, interventional, randomized, and non-randomized; interventions were biological, behavioral, dietary supplement, device, procedure, drug, gene transfer, other, and not specified.

After a trial is completed, time is needed to analyze the data and write the paper. So we conducted a sensitivity analysis in publication rate by excluding trials completed after July 2008 from our analysis.

We also calculated the registration number reporting rate for the publication full texts.

Statistical analysis

Publication rates were described as proportions. Data were analyzed by EXCEL software.

Results

Search results

Number of registered trials.

We initially screened 1294 records of Chinese trials in clinicaltrials.gov. After we carefully read the registration information, we excluded 551 records with a recruitment location in China, but no investigators affiliated with China administratively.

We finally included 1247 records (743 in clinicaltrials.gov, 428 in ChiCTR, 55 in ISRCTN.COM, and 21 in ACTR). No records Chinese trials were found in the Indian, German, Japanese, Dutch, Sri Lankan, Iranian, or Pan African registries.

Result publication.

As of July 14, 2009, 35.5% (443/1247) of the registered Chinese trials had been completed. The publication rate of the completed trials was 35.2% (156/443). The publication rate of the completed trials in ACTR was the highest of the 4 registries, and similar publication rates in ChiCTR and clinicaltrials.gov were found. As of July 14, 2008, 332 trials were completed. The publication rate was a little higher at 39.8% than that of the trials completed before July 14, 2009 (see Table 1). Among the 145 completed trials in ChiCTR, we found the published results for 40 through our systematic search. We called the investigators of the other 105 trials. A total of 51 trial investigators were contacted; the other 54 investigators could not be contacted because the telephone number in the registry was wrong, the mobile phone was off, or there was no answer. The investigator of one trial stated that the results were published but provided no details. We did not find the publication.

thumbnail

Table 1. Publication rate of the completed Chinese trials in WHO/ICMJE primary registries.

doi:10.1371/journal.pone.0012676.t001

The call investigation found result publications for 13 trials. The main reasons investigators gave for non-publication were that the manuscript was in preparation (21 trials), that there was no intention to publish results (7 trials), or that the manuscript had been rejected by journals (2 trials). The investigators of 7 trials said that although the trials were indexed as “completed,” these trial were not in fact completed.

We could not get the email or telephone number of the investigators for the completed Chinese trials in clinicaltrials.gov because contact information is only displayed in that registry when the study is recruiting subjects. We obtained all the full texts of the included trials in ACTR and ISRCTN. Therefore, we did not contact the investigators for trials in these three registries.

Factors association with the result publication

Sponsorship.

The publication rate for the Chinese trials sponsored by industry was the lowest of the four sponsor types at 23.8%, while the publication rate for Chinese trials sponsored by universities was the highest, at 40.7% (table 2).

thumbnail

Table 2. The publication rate and sponsorship of the completed Chinese trials in ChiCTR, clinicaltrials.gov, ISRCTN, ACTRN.

doi:10.1371/journal.pone.0012676.t002

The publication rate for trials sponsored by industry in ChiCTR was higher than that of trials sponsored by governments or hospitals (table 2). However, the available sample was small (only 2 trials were sponsored by industry, with 1 published and 1 unpublished result). The publication rates for government-sponsored trials in ChiCTR, clinicaltrials.gov, and ISRCTN were all low (table 2).

Publication rate by trial stage.

Of the 743 Chinese trials in clinicaltrials.gov, 245 (33.0%) had been completed, 23 (3.1%) had been terminated, 5 (0.7%) had been suspended, and 470 (63.2%) were ongoing. The publication rate of the competed, ongoing, suspended, and terminated trials was 36.3% (89/245), 3.6% (17/470), 10% (1/5), and 8.7% (2/23), respectively.

Of the 145 completed trials in ChiCTR, 124 (85.5%) were retrospectively registered (registered between trial start and result publication), and 21 (29.8%) were prospectively registered. The publication rate of retrospectively registered trials (29.8%) was higher than that of prospectively registered trials (14.3%).

Publication rate by study design.

The publication rate of randomized controlled trials was higher than those of cohort and case-control studies. The result publication rates for trials of preventative and health services research were higher than those of supportive care, treatment and diagnostic trials. The publication rate of interventional and observational studies was similar (Table 3).In Clinicaltrials.gov, publication rate of randomized controlled trials was higher than non-randomized controlled trials, publication rate of interventional studies was higher than observational studies; while in ChiCTR and ACTRN, it was contrary.

thumbnail

Table 3. Publication rate of study type and design in clinicaltrials.gov (n, %).

doi:10.1371/journal.pone.0012676.t003
Publication rate by intervention type.

No classification of the interventions of trials was provided in ChiCTR, so we only analyzed the publication rates for different interventions in clinicaltrials.gov. The publication rate for biological research was the highest (Table 4). The number of the drug trial was the most, but its publication rate was the lowest (Table 4).

thumbnail

Table 4. The publication rate by interventions for Chinese trials in clinicaltrials.gov (n,%).

doi:10.1371/journal.pone.0012676.t004

Registration number reporting in full text of result publications

We obtained the full text of result publications for 62.2% (97/156) of the completed trials. Only 25.8% (25/97) of the result publications reported registration number.

Of the 38 result publications in ChiCTR, Only 1 (a doctoral dissertation) reported a registration number.

Of the 45 result publications in clinicaltrials.gov, 21publications for 20 trials reported registration number. Apart from one result publication in the Chinese-language National Medical Journal of China, the publications that reported registration number were all published in English-language journals outside China.

Nine publication full texts for the 9 trials in ISRCTN were obtained. Only two publications for 2 trials reported registration numbers. Five publications for 5 trials reported informed consent and the name of the ethical review body.

We obtained 5 publication full texts for 5 trials in ACTR. Two of the publications reported registration numbers.

Discussion

The publication rate of completed, registered Chinese trials was only35.2%.Even for trials had been completed since at least one year, the publication rate was also only 39.8%. The publication rate of registered Chinese trials is lower than the average (46%) for all trials registered in clinicaltrials.gov. The publication rate is similar in clinicaltrials.gov and ChiCTR. It was also similar to the low publication rates found in previous studies [23]. A meta-analysis showed that the publication rates for trials in inception cohort, regulatory cohort, abstract cohort, and manuscript cohort varied from 60% to 93% for significant results and from 20% to 86% for non significant results [23].

In 2005 ICMJE recommended that investigators be required to prospectively register randomized controlled trials, but only a few Chinese journals have implemented this policy. Accordingly, we found that only 25.8% of result publications reported a registration number. In contrast, Mathieu et al. found recently that among articles indexed by MEDLINE in cardiology, rheumatology, and gastroenterology, 76.4% reported the registration number [24]. This may be because higher impact journals are more likely to require registration number reporting. Therefore, we suggest that all journals should require submitted papers to include a registration number.

Factors association with non publication

The low publication rate we found indicates that the current trial registration system does not promote publication effectively. It does, however, provide a convenient means to identify non publication. We found several factors associated with non publication:

Sponsorship.

Trials sponsored by industry were less likely to have result publications than trials with other funding sources, indicating that conflict of interest may be a factor in non publication. This accords with the findings of Ross et al., who report that the publication rate of trials sponsored by industry is lower than those with non-industry support (40% versus 56%) in clinicaltrials.gov. However, Ross et al. found that the publication rate for trials primarily sponsored by the National Institutes of Health (NIH) is also just 40% (30/74) [15]. In contrast, Dickersin et al. report that the publication rate for 198 clinical trials funded by the NIH is 93%[21]. Ramsey et al. find that just 5.9% of the oncology trials sponsored by industry in clinicaltrials.gov published results indexed by PubMed[25]. Liebeskind et al. report that the publication rate of trials on efficacy of acute stroke treatments conducted between 1955 and 1999 is at 76% (136/178)[26]. A systematic review of 6 studies of funding sources and publication status shows that trials funded by industry are less likely published than other funded sources [27].

Study design.

We find that the publication rate for randomized controlled trials is higher than for case control and cohort studies. Easterbrook et al. also find that randomized controlled trials have a lower risk of non publication than observational studies [20].

Intervention type.

Though drug trials are the most numerous among those registered, their publication rate is lower than those of biological, behavioral, other, dietary supplement, and device procedures. This suggests that the publication bias for drug trials is more serious than for other interventional trials in China. A similar effect is also found in clinical trials involving acute stroke[26].

Termination/suspension.

Only about one-tenth of terminated or suspended trials had result publications, suggesting a high risk of non publication for these trials.

In our telephone interviews, the reasons cited by investigators for non publication included that the manuscript had been rejected by the journal, and a lack of time for writing. This is consistent with previous reports. Kryzyzanowska et al. interviewed 34 authors who had not published results, and found that common reasons for failure to publish were lack of time, funds, or other resources [22]. Other studies show that the main reason for failure to publish results is that investigators lack the time to submit the full manuscript to a peer-reviewed journal [21], [28][30].

Reducing non publication

The Food and Drug Administration Amendment Act of 2007 requires investigators to submit results to clinicaltrials.gov. Investigators who do not comply can be fined as much as $10,000 per day or have their federal research funds withheld or recovered[31]. The law may work. The publication rate of trials sponsored by NIH (46%) is higher in clinicaltrials.gov[15] than that of Chinese trials (35%) registered in WHO primary registries in our study.

A recent study concludes that three proposed remedies, financial disclosure, reporting standards, and trial registries, failed to control sponsorship bias[32]. The low publication rate of Chinese trials registered in clinicaltrials.gov or ChiCTR suggests that trial registration alone, without a requirement of result publication, does not improve publication rate. Non publication was still common in trials registered in clinicaltrials.gov [15], [24].

Although trial registration is becoming more common in China, no systematic enforcement of the result posting requirement has yet been established for WHO primary registries. Posting results in the registry and publishing in peer reviewed journals should be encouraged to avoid a low result publication rate.

Limitations

Our study has some limitations. First, it only investigates the publication rate of registered trials, so the time lag between trial and publication, as well as any selective outcome reporting bias, remain uncertain. Second, we are unable to follow up with all of the lead investigators on completed, unpublished trials, since no email addresses or telephone numbers are posted for completed Chinese trials in clinicaltrials.gov, and investigators of 54 trials in ChiCTR could not be contacted. So the publication status and reasons for non publication of these trials is unknown. Third, although we use a comprehensive search strategy to find the result publications, some publications are likely omitted. In order to more fully understand publication status in registered Chinese trials, further research will be need to examine the time lag between trial and publication, and the comparative publication rates for positive and negative results.

Conclusion

Many previous studies have examined potential sources of non publication, abbreviated publication, selective outcomes reporting, and time lag bias[26], [29][31], [32]. Just two studies[15], [31] have investigated publication rate through registration[15], [32]. This is the first study to report on the publication rate of registered Chinese trials.

Non publication trial is a more serious problem in China than in other countries. Mandatory result posting in a registry is needed to promote result publication. Result publication policies should especially focus on result publication of industry-sponsored trials, poorly designed trials, and terminated or suspended trials. Regular, complete, and transparent reporting is urgently needed to ensure the integrity of Chinese science.

Acknowledgments

We thank Mr. He Lin in Library of West China Hospital for helping search for full texts, and Ms. Shawna Williams in West China Medical School for helping edit this article.

Author Contributions

Conceived and designed the experiments: LX LY. Performed the experiments: LX YS SS. Analyzed the data: LX YS SS. Contributed reagents/materials/analysis tools: LX YS. Wrote the paper: LX LY.

References

  1. 1. Abaid LN, Grimes DA, Schulz KF (2007) Reducing publication bias of prospective clinical trials through trial registration. Contraception 76: 339–341.
  2. 2. Al-Marzouki S, Roberts I, Evans S, Marshall T (2008) Selective reporting in clinical trials: Analysis of trial protocols accepted by The Lancet. Lancet 372: 201.
  3. 3. Cerivastatin CA (2001) Which lessons for clinical practice? Recenti Prog Med 92: 641–643.
  4. 4. Chan AW, Altman DG (2005) Epidemiology and reporting of randomized trials published in PubMed journals. Lancet 365: 1159–1162.
  5. 5. Chan AW, Hróbjartsson A, Haahr MT, Gotzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: Comparison of protocols to published articles. JAMA 291: 2457–2465.
  6. 6. Ghersi D, Pang T (2009) From Mexico to Mali: four years in the history of clinical trial registration. Journal of Evidence-Based Medicine 2: 1–7.
  7. 7. Turner EH, Matthews AM, Linardatos E, Tell RA, Rosenthal R (2008) Selective publication of antidepressant trials and its influence on apparent efficacy. N Engl J Med 358: 252–260.
  8. 8. Wessely S, Kerwin R (2004) Suicide risk and the SSRIs. JAMA 292: 379–381.
  9. 9. Yong MS, Ioannidis JPA, Al-ubaydli O (2008) Why current publication practices may distort science. PLoS Medicine 5: 1418–1422.
  10. 10. De Angelis C, Drazen JM, Frizelle FA, Haug C, Hoey J, et al. International Committee of Medical Journal Editors (2004) Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA 292: 1363–1364.
  11. 11. Laine C, Horton R, DeAngelis C, Drazen JM, Frizelle FA, et al. International Committee of Medical Journal Editors (2007) Clinical trial registration: looking back and moving ahead. Lancet 369: 1909–1911.
  12. 12. Krleza-Jerić K (2005) Clinical trial registration: the differing views of industry, the WHO, and the Ottawa Group. PLoS Med 2: e378.
  13. 13. Zarin DA, Tse T (2008) Moving Towards Transparency of Clinical Trials. Science 319: 1340–1342.
  14. 14. World Health Organization.International Clinical Trials Registry Platform. Available: http://www.who.int/ictrp/about/details/e​n/index.html. Accessed 22 December 2008.
  15. 15. Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM (2009) Trial Publication after registration in ClinicalTrials.Gov: A cross-sectional analysis. PLoS Medicine 6: e1000144. doi:10.1371/journal.pmed.1000144.
  16. 16. Liu XM, Li YP, Yu XT, Feng J, Zhong XS, et al. (2009) Assessment of registration quality of trials sponsored by China. Journal of Evidence-Based Medicine 2: 8–18.
  17. 17. Jia H (2010) China beckons to clinical trial sponsors. Nature Biotechnology. Available http://www.nature.com/drugdisc/news/arti​cles/nbt0705-768.html. Accessed 9 March 2010.
  18. 18. Young NS, Ioannidis JPA, Al-Ubaydli O (2008) Why current publication practices may distort science. PLoS Medicine 5: e201. doi:10.1371/journal.pmed.0050201.
  19. 19. Ghersi D, Clarke M, Berlin J, Gülmezoglu AM, Kush R, et al. (2008) Reporting the findings of clinical trials: a discussion paper. Bulletin of the World Health Organization 86: 492–493.
  20. 20. Easterbrook PJ, Berlin JA, Gopalan R, Mattews DR (1991) Publication bias in clinical research. Lancet, 337: 867–872.
  21. 21. Dickersin K, Min Y (1993) NIH clinical trials and publications bias. Online J Curr Clin Trials Doc No50:
  22. 22. Krztzabiwska MK, Pintilie M, Tannock IF (2010) Factors associated with failure to publish large randomized trials presented at an oncology meeting. JAMA 290: 495–501.
  23. 23. Song FJ, Parekh-Bhurke S, Hooper L, Loke YK, Ryder JJ, et al. (2009) Extent of publication bias in different categories of research cohorts: a meta-analysis of empirical studies. BMC Medical research methodology 9: 79. doi:10.1186/1471-2288-9-79 .
  24. 24. Mathieu S, Boutron I, David M, Altman DG, Ravaud P (2009) Comparison of registered and published primary outcomes in randomized controlled trials. JAMA 302: 977–984.
  25. 25. Ramsey S, Scoggins J (2008) Commentary: Practicing on the Tip of an Information Iceberg? evidence of underpublication of registered clinical trials in oncology. The oncologist 13: 925–929.
  26. 26. Liebeskind DS, Kidwell CS, Sayre JW, Saver JL (2006) Evidence of publication bias in reporting acute stroke clinical trials. Neurology 67: 973–979.
  27. 27. Lexchin J, Bero LA, Djulbegovis B, Clark O (2003) Pharmaceutical industry sponsorship and research outcome and quality:systematic review. BMJ 326: 1167–1177.
  28. 28. Weber EJ, Callaham ML, Wears RL, Barton C, Young G (1998) Unpublished research form a medical specialty meeting: why inverstigators fail to publish. JAMA 280: 257–259.
  29. 29. De Bellefeuille C, Morrison CA, Tannock IF (1992) The fate of abstracts submitted to a cancer meeting: factors which influence presentation and subsequent publication. Ann Oncol 3: 187–191.
  30. 30. Timmer A, Hilsden RJ, Cole J, Hailey D, Sutherland LR (2007) Publication bias in gastroenterological research____a retrospective cohort study based on abstract submitted to a scientific meeting. BMC Med Res Methodol 2: 7.
  31. 31. Tuma RS (2010) New law may be having some effect publication bias. JNCI 102: 290–292.
  32. 32. Doucet M, Sismondo S (2008) Evaluating solutions to sponsorship bias. J Med Ethics 34: 627–630. doi-10.1136/jme.2007.022467.