Our perception of clinical trial outcomes may be distorted
European and national legislation clearly specify the mandatory structure and content of a clinical trial protocol, e.g. for a randomised, controlled phase II human drug study [1-4]. The same applies to the structure and content of a clinical study report [5]. One should assume that a publication of a given clinical trial fulfilling these legal requirements is a high-quality and complete mirror of the trial protocol and the study report. The reality is different. Read on to learn more and discover what measures are being taken by the scientific community to avoid distorting public perception and reporting bias by implementing new rules on publishing results of clinical trials.
Many reviews, have pointed out insufficient and inaccurate reporting of clinical trials. Analyses of PubMed-indexed trial reports in 2006 revealed that information on participants’ assignment to comparison groups was provided in no more than 34% of the cases. Only 53% of reports defined a primary endpoint, and only 45% reported a sample size calculation [6]. This led to an update of the CONSORT (Consolidated Standards of Reporting Trials) statement [7], an evidence-based, minimum set of recommendations for reporting randomised controlled trials (RCTs). It includes a 25-essential-items checklist with a focus on trial design, analysis and interpretation – and a flow-diagram to display the progress of all trial participants through the trial [7]. The primary objective of CONSORT is to provide guidance to authors how to improve reporting of their trials, and to assist readers, peer reviewers and editors to critically appraise and interpret reports of RCTs [8].
Meanwhile, extensions to the CONSORT statement have been published in order to cover a wider range of trial designs, such as, non-inferiority and equivalence -, cluster - and pragmatic trials [9-11]. Another extension deals with the reporting of abstracts [12]. Additional statements are available including reporting of observational studies (STROBE) [13], systematic reviews and meta-analyses (PRISMA; PRISMA-IPD) [14, 15], and diagnostic accuracy studies (STARD 2015) [16]. Reporting of harms and adverse events has been shown still to be inadequate [17, 18] – this has led to two further statement extensions: the CONSORT for harms checklist [19] and the PRISMA harms checklist [20]. The SPIRIT 2013 Explanation and Elaboration Paper [21] extends guidance to clinical trial protocols. The AGREE Reporting Checklist comprises a tool to improve reporting of clinical practice guidelines [22]. The list could still be continued. The EQUATOR (Enhancing the QUAlity and Transparency Of health Research) network, which was formed to support the development and dissemination of all these guidelines, currently includes 313 reporting guidelines on its website [23].
Potential sources of bias in publications of clinical trials
The Cochrane Collaboration published its tool for assessing risk of bias in randomised trials and specified six bias domains: selection bias, performance bias, detection bias, attrition bias, reporting bias and other bias [24].
Sources of selection bias (= biased allocation to interventions) include inadequate random sequence generation and allocation concealment. Performance bias could be due to non-blinding of participants and personnel, and detection bias due to knowledge of the allocated interventions by outcome assessment – to be avoided by blinding of outcome assessment. Attrition bias refers to incomplete outcome data and could be caused by their amount, nature or handling. Reporting bias implies selective outcome reporting and other bias could be due to problems not covered elsewhere. In order to judge a given trial or manuscript each of the above mentioned potential sources of bias should be classified as ”low risk” / “high risk” /”unclear risk” [24].
Unpublished and abandoned trials distort the real effect of novel drugs
Reporting bias due to unpublished clinical trial results, delayed publication or under-reporting of adverse event data - in case of negative or unfavorable results - may distort the real effect of an intervention or a new drug, may harm patients and is in contrast to the ethical principles set in the Declaration of Helsinki [25, 26]. It is a challenge to search for unpublished clinical trial evidence [27], and there is an ongoing request to register all trials and publish the results [28, 29].
Sharing Clinical Trial Data
The European Medicines Agency (EMA) has adopted a policy on publication of clinical data [30], and The International Committee of Medical Journal Editors (ICMJE) recently made a proposal concerning data-sharing requirements and asked for feedback by April 18, 2016 [31]. They propose to require authors to share with others the de-identified individual-patient data (IPD) underlying their trial results – for transparency and reproducibility reasons - and would like them to include a data sharing plan as a component of trial registration. ClinicalTrials.gov has already added a respective element to its registration platform [31].
Summary
In summary, some lessons have already been learnt concerning improved reporting of clinical trial results – and tools how to do so are available. A lot more, however, still needs to be done to achieve transparent, accurate and complete publications of each clinical trial including sharing of data with other researchers. Continuous careful observation of this process - accompanied by further publications on which next steps have been reached and what has failed – will be necessary.
References
[1] European Medicines Agency. ICH Topic E6 (R1) Guideline for Good Clinical Practice. Step 5. Note for Guidance on Good Clinical Practice (CPMP/ICH/135/95). July 2002.
[2] Richtlinie 2001/20/EG des Europäischen Parlaments und des Rates vom 4 April 2001 zur Angleichung der Rechts- und Verwaltungsvorschriften der Mitgliedstaaten über die Anwendung der guten klinischen Praxis bei der Durchführung von klinischen Prüfungen mit Humanarzneimitteln. Amtsblatt der Europäischen Gemeinschaften 1.5.2001.
[3] Arzneimittelgesetz in der Fassung der Bekanntmachung vom 12. Dezember 2005 (BGBI. I S. 3394), das durch Artikel 3 des Gesetzes vom 4. April 2016 (BGBI. I S. 569) geändert worden ist.
[4] GCP-Verordnung vom 9. August 2004 (BGBI. I S. 2081), die zuletzt durch Artikel 8 des Gesetzes vom 19. Oktober 2012 (BGBI. I S. 2192) geändert worden ist.
[5] European Medicines Agency. ICH Topic E 3 Structure and Content of Clinical Study Reports. Step 5. Note for Guidance on Structure and Content of Clinical Study Reports (CPMP/ICH/137/95). July 1996.
[6] Hopewell S, Dutton S, Yu LM, Chan AW, Altman DG. The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed. BMJ 2010; 340: c723.
[7] Schulz KF, Altman DG, Moher D, for the CONSORT Group. CONSORT 2010 Statement: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c332.
[8] Moher D, Hopewell S, Schulz KF, et al. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 2010; 340: c869.
[9] Piaggio G, Elbourne DR, Pocock SJ, Evans SJW, Altman DG, for the CONSORT Group. Reporting of noninferiority and equivalence randomized trials. Extension of the CONSORT 2010 statement. JAMA 2012; 308: 2594-2604.
[10] Campbell MK, Piaggio G, Elbourne DR, Altman DG, for the CONSORT Group. Consort 2010 statement: extension to cluster randomised trials. BMJ 2012; 345: e5661.
[11] Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D, for the CONSORT and Pragmatic Trials in Healthcare (Practihc) groups. BMJ 2008; 337: a2390
[12] Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF, and the CONSORT Group. CONSORT for reporting randomised trials in journals and conference abstracts. Lancet January 22, 2008. DOI:10.1016/S0140-6736(07)61835-2.
[13] von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, for the STROBE Initiative. The strengthening the reporting of observational studies in epidemiology (STROBE) statement: guidelines for reporting observational studies. Int J Surg 2014; 12: 1495-1499.
[14] Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Medicine 2009; 6: e1000097.
[15] Stewart LA, Clarke M, Rovers M, Riley RD, Simmonds M, Stewart G, Tierney JF, for the PRISMA-IPD Development Group. Preferred reporting items for a systematic review and meta-analysis of individual participant data. The PRISMA-IPD statement. JAMA 2015; 313: 1657-1665.
[16] Bossuyt PM,Reitsma JB, Bruns DE, Gatsonis CA, Glasziou PP, Irwig L, Lijmer JG, Moher D, Rennie D, de Vet HCW, Kressel HY, Rifai N, Golub RM, Altman DG, Hooft L, Korevaar DA, Cohen JF, for the STARD Group. BMJ 2015; 351: h5527.
[17] Zorzela L, Golder S, Liu Y, Pilkington K, Hartling L, Joffe A, Loke Y, Vohra S. Quality of reporting in systematic reviews of adverse events: systematic review. BMJ 2014; 348:f7668.
[18] Saini P, Loke YK, Gamble C, Altman DG, Williamson PR, Kirkham JJ. Selective reporting bias of harm outcomes within studies: findings from a cohort of systematic reviews. BMJ 2014; 349: g6501.
[19] Ioannidis JPA, Evans SJW, Gøtzsche PC, O’Neill RT, Altman DG, Schulz K, Moher D, for the CONSORT Group. Better reporting of harnms in randomized trials: an extension of the CONSORT statement. Ann Intern Med 2004; 141: 781-788.
[20] Zorzela L, Loke YK, Ioannidis JP, Golder S, Santaguida P, Altman DG, Moher D, Vohra S, PRISMA harms group. PRISMA harms checklist: improving harms reporting in systematic reviews. BMJ 2016; 352: i157.
[21] Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbartsson A, Schulz KF, Parulekar WR, Krleža-Jeric K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013; 346: e7586.
[22] Brouwers MC, Kerkvliet K, Spithoff K, AGREE Next Steps Consortium. The AGREE reporting checklist: a tool to improve reporting of clinical practice guidelines. BMJ 2016; 352: i1152.
[23] Equator Network. http://www.equator-network.org. Accessed April 26, 2016.
[24] Higgins JPT, Altman DG, Gøtzsche PC, Jüni P, Moher D, Oxman AD, Savović J, Schulz KF, Weeks L, Sterne JAC, Cochrane Bias Methods Group, Cochrane Statistical Methods Group. The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 2011; 343: d5928.
[25] Wolfe SM. Selective clinical trial reporting: betraying trial participants, harming patients. Reporting bias found in trials of cardiovascular devices. BMJ 2015; 350: h2753.
[26] World Medical Association, Inc. WMA Declaration of Helsinki – Ethical principles for medical research involving human subjects. Adopted by the 18th WMA Assembly, Helsinki, Finland, June 1964 and amended by the: 64th WMA Assembly, Fortaleza, Brazil, October 2013.
[27] Chan AW. Out of sight but not out of mind: how to search for unpublished clinical trial evidence. BMJ 2011; 344: d8013.
[28] Chalmers I, Glasziou P, Godlee F. All trials must be registered and the results published. BMJ 2013; 346: f105.
[29] Doshi P, Dickersin K, Healy D, Vedula SS, Jefferson T. Restoring invisible and abandoned trials: a call for people to publish the findings. BMJ 2013; 346: f2865.
[30] European Medicines Agency. European Medicines Agency polica on publication of clinical data for medicinal products for human unse. EMA/240810/2013. 2 October 2014.
[31] Taichman DB, Backus J, Baethge C, Bauchner H, De Leeuw PW, Drazen JM, Fletcher J, Frizelle FA, Groves T, Haileamlak A, James A, Laine C, Peiperl L, Pinborg A, Sahni P, Wu S. Sharing clinical trial data. A proposal from the International Committee of Medical Journal Editors. JAMA 2016; 315: 467-468.