Science is a dynamic process where conclusions evolve as new evidence is generated. As such, science is thought to correct itself over time, although how efficiently it does this is debatable, with some errors persisting longer than others. It is vital to correct the scientific record to maintain its integrity, and this can be done on a collective level or through correction of one’s own work. But why might researchers lose confidence in their previously published findings, and how often do they voluntarily correct their own errors?

In a recent LSE blog, Dr Julia Rohrer discusses her project on the subject of self-correction. In the study, published in Perspectives on Psychological Science, Dr Rohrer and colleagues invited psychological researchers to submit statements describing conclusions that they no longer trust because of their own mistakes. The authors received 13 statements spanning a wide range of psychological domains and methods, in which the reasons for the loss of confidence could broadly be categorised as:

• Methodological error: misspecified statistical models or programming error.
• Invalid inference: discrepancy between results and claims.
• P-hacking: failing to properly account for degrees of freedom when conducting or reporting analyses, stemming from a poor understanding of relevant statistical considerations.

The authors also conducted a cross-disciplinary online survey, in which 44% of the 316 respondents reported losing confidence in at least one of their published findings – 52% of whom because of self-admitted, questionable research practices, such as selective reporting. The loss of confidence was publicly disclosed in only 17% of cases, primarily by way of a statement in a later publication, conference presentation, or social media post. Reasons for not self-correcting included:

• feeling insufficiently sure about the subject matter to proceed
• deeming it unnecessary as the finding did not attract much attention
• not wanting to upset co-authors
• not knowing an appropriate venue
• concerns about how the loss of confidence would be perceived.

Dr Rohrer and colleagues concluded that it is necessary to tackle current obstacles to individual self-correction, so that admitting errors is destigmatised and becomes a routine part of science. The authors suggest that a consistent, open, and transparent corrections process is required that directly links any amendments to the original research article.

44% of the 316 respondents reported losing confidence in at least one of their published findings – 52% of whom because of self-admitted, questionable research practices, such as selective reporting. The loss of confidence was publicly disclosed in only 17% of cases.

Moving forward, a more radical approach may be needed to create a research culture more conducive to self-correction where publications can be updated, and old versions archived, as errors or new information come to light. Preprint servers already allow authors to disclose additional information to supplement their published manuscripts, while the Springer Nature Living Reviews journals facilitate updates to their articles to incorporate the latest developments. We look forward to seeing how other journals and publishers might address this issue.

—————————————————–

—————————————————–