KEY TAKEAWAYS

• Researchers have raised concerns about the reproducibility of AI-based studies across many scientific fields including medicine.
• Reporting checklists and guidelines could help to avoid common pitfalls in studies using AI but more needs to be done to ensure scientific credibility.

The results of many studies that use machine learning or artificial intelligence (AI) methodologies could be being overstated. This warning of a reproducibility crisis in machine learning was recently reported by Elizabeth Gibney in Nature, and is based in part on the findings of a preprint co-authored by Sayash Kapoor and Arvind Narayanan, who identified a collective 329 studies across multiple scientific disciplines with shortcomings regarding the reproducibility of their findings.

Machine learning and AI have become powerful tools at the disposal of biomedical researchers, but the reproducibility of these methodologies and associated outcomes is paramount for their credibility. A methodologic pitfall frequently encountered by Kapoor and Narayanan in their analysis was so called ‘data leakage’, where data used to train the AI model were subsequently used in the test data set, potentially exaggerating the AI’s ability to make accurate predictions. To counter this, Kapoor and Narayanan propose that researchers use ‘model info sheets’ to transparently report the details of their AI models.

“Unless we do something like this, each field will continue to find these [reproducibility] problems over and over again.”

Reporting checklists are not unfamiliar to AI researchers in the biomedical field, as noted by Gibney, who referred to initiatives like the EQUATOR Network’s CONSORT-AI and SPIRIT-AI reporting guidelines developed by Dr Xiao Liu and colleagues. While checklists are an important and useful tool, greater collaboration between researchers and specialists in machine learning could also help. It is encouraging then to note the 1,200 people registering to attend a workshop on reproducibility co-organised by Kapoor with the mission to resolve the reproducibility crisis in AI-based science.

—————————————————–