Fixing the problem of outcome-switching in clinical trials
It is established best-practice that all clinical trial outcomes should be pre-specified in a trial protocol and on a clinical trial registry and then reported once the trial is complete. To prevent ‘data-dredging’ and the delivery of misleading results, differences between reported outcomes and those pre-specified should be declared along with an explanation of the timing and reason for the change.
Despite the majority of major medical journals endorsing trial reporting guidelines such as CONSORT, the problem of outcome-switching is prevailing. Ben Goldacre and his group at the Centre for Evidence-Based Medicine (CEBM) at the University of Oxford are determined to fix this. The COMPare (CEBM Outcome Monitoring Project) team are monitoring every trial published in the top five medical journals, comparing the outcomes reported with the outcomes pre-specified in protocols or trial registries. They have found that most have discrepancies (on average, each trial reported 62.0% of its specified outcomes and silently added 5.3 new ones). They have submitted correction letters to the publishing journal for every misreported trial. While some journals, such as the BMJ, have demonstrated best practice by quickly responding with a published correction on the trial in question, others have not.
In his recent column published in Nature, Ben Goldacre concludes that “journal editors now need to engage in a serious public discussion on why this [outcome-switching] is still happening”. He proposes that audit and accountability are the key to best-practice in medical research.
You should be aware that Goldacre et al are ignoring protocol amendments, and comparing the reported outcomes only to protocols dated prior to the trial start date. The only time amended protocols are considered is when the authors have remembered to describe the protocol amendment in the paper (CONSORT item 6b).