KEY TAKEAWAYS

• Approximately 20% of clinical trials are thought to be false.
• Cochrane published guidance on how to manage potentially problematic studies in their systematic literature reviews.
• Dr Richard Smith suggests it’s time to assume all trials are untrustworthy unless proven otherwise.

With increasing evidence that scientific fraud is widespread, Cochrane has published a policy for managing untrustworthy clinical trials in the context of systematic literature reviews. However, Dr Richard Smith, cofounder of the Committee on Medical Ethics (COPE) and member of the board of the UK Research Integrity Office, suggests that it is time to go a step further and assume that all research is fraudulent until proven otherwise.

In a BMJ opinion piece, Smith outlines evidence from research leaders who, in their own investigations, found that many studies underlying systematic reviews were fatally flawed or contained false data. Professor Ben Mol, leader of the Evidence-Based Women’s Health Care Research Group at Monash University, estimates that 20% of trials are false. Availability of individual patient data increases the likelihood of detecting fraud, with one study showing that up to 44% of examined trials were untrustworthy.

Cochrane’s policy provides guidance for dealing with these ‘potentially problematic’ trials, including:

• retracted studies
• studies with a published Expression of Concern
• studies where there are serious questions about trustworthiness of data or findings but no formal post-publication amendment.

However, as noted in an editorial accompanying the policy, the scope of problematic studies is wide-ranging and there is no validated method to identify them (although tools such as the REAPPRAISED checklist can be useful). As more evidence becomes available and consensus emerges in this area, the guidance will need to be updated.

The scope of problematic studies is wide-ranging and there is no validated method to identify them.

With the risk of medical research fraud ultimately leading to patients being given inappropriate treatment, Smith concluded that it may be time to move away from trusting research is honest and reliable to assuming it is untrustworthy until there is evidence to the contrary.

—————————————————–