Artificial intelligence (AI) is a rapidly evolving field that has many potential health applications, including systems that can be used for screening and triage, diagnosis, decision support and treatment recommendation. Concerns regarding the design and reporting of recent trials evaluating such systems have led to the development of an extension to the SPIRIT (The Standard Protocol Items: Recommendations for Interventional Trials) 2013 statement, an international guideline aiming to standardise the quality and completeness of published trial protocols. Details of the extension and the methods used in its development were recently published by Samantha Cruz Rivera et al in the British Medical Journal.

The SPIRIT-AI extension, which complements the CONSORT-AI guidelines for the reporting of AI trials, provides consensus-based, AI-specific guidance on what should be included in the trial protocol, and should be used alongside SPIRIT 2013 and other relevant extensions. Items recommended for inclusion in the new checklist include:

• algorithm version
• input and output data
• expertise of users
• plans to identify and analyse performance errors
• prior level of evidence for validation of the AI intervention.

The latter two points were added to address safety concerns raised by the consensus group in recognition that that AI systems may unpredictably produce errors that are not easily detectable or explainable by human judgment, which could have unintended but harmful consequences.

Furthermore, as some AI systems continuously learn from new data their safety performance can evolve and change over time. While the issue of continuously evolving systems has not been covered in the current SPIRIT-AI extension, the authors suggest that software changes should be identified and documented and note that this topic may be addressed in future versions of SPIRIT-AI. Due to the relative newness of the field, there were few published AI trials and no published AI trial protocols at the time that the SPIRIT-AI guidance was developed. The authors acknowledge the importance of re-evaluating and updating SPIRIT-AI as AI technology continues to evolve and as its application expands to include therapeutic uses in addition to detection and diagnostics.

After reading the SPIRIT-AI article, click here for a brief survey and to receive your authorization code for your Credit Tracker. This serves as documentation for the activity.

After reading the CONSORT-AI article, click here for a brief survey and to receive your authorization code for your Credit Tracker. This serves as documentation for the activity.

——————————————————–

Summary by Alice Wareham PhD, CMPP from Aspire Scientific

——————————————————–

With thanks to our sponsor, Aspire Scientific Ltd