It has been suggested that a sizeable number of registered clinical trials fail to report their results, and that negative or null results are especially likely to remain unpublished. To help tackle the so-called file drawer effect, Ben Goldacre and Anna Powell-Smith have developed an automated tool, TrialsTracker, to flag up trials with unreported results.

Goldacre and Powell-Smith, who are based at the University of Oxford, examined the fate of almost 26,000 trials that have had a status of ‘completed’ on ClinicalTrials.gov for at least two years. Using the unique registry ID number assigned to each trial, TrialsTracker searched for results on ClinicalTrials.gov and in PubMed. For 45.2% of registered trials, no such data could be found. The output from TrialsTracker is available online, enabling users to identify those sponsors that appear to have the greatest number and proportion of trials with missing data. Full details of the study have been published on the open access platform F1000Research.

Goldacre and Powell-Smith acknowledge that automated searches may be less precise than manual searches. Publications that omit the ClinicalTrials.gov registry identifier may be overlooked, while reports that include the registry identifier but do not present full data may be erroneously scored as compliant. On the other hand, Goldacre and Powell-Smith argue that results obtained with TrialsTracker are consistent with those from previously published manual searches. They emphasise too that automation allows the analysis to be repeated regularly and the results updated, creating an incentive for sponsors to improve their standing in the tables.

Nevertheless, commentators have criticised the methodology used by TrialsTracker. In a rapid response published in the BMJ this week, Adam Jacobs has flagged imperfections with the algorithm. He downloaded the Trials Tracker raw data and looked up the first 10 “undisclosed” trials. All except 2 of those trials had their results disclosed on the sponsoring company’s own website. Of the remaining 2, one was published in a peer-reviewed journal, and only one remained undisclosed after a further quick (5 minute) search. This “undisclosed” trial was of a drug that was abandoned in clinical development in 2008. He therefore concludes that the algorithm used by TrialsTracker “…may over-estimate the extent of non-disclosure by a factor of 10”.

——————————————————–

Summary by Louisa Lyon, DPhil from Aspire Scientific.