Standards Development

Scope

This document provides guidance for applying an ISO 14971 risk management process when evaluating medical technology utilizing machine learning (ML). It is intended to apply to ML-enabled medical devices throughout all phases of the product lifecycle.

This document is intended to be used in conjunction with ISO 14971. It does not modify the ISO 14971 risk management process—rather it provides information and guidance to inform the application of ISO 14971 to ML medical technology.

This document addresses the same types of risk that are addressed in ISO 14971 but focuses on risks that are elevated with or unique to ML medical devices. Because artificial intelligence (AI) and ML are software-driven, the unique or elevated risks are those around data management, feature extraction, algorithm training, evaluation, bias, health inequity, safety, and cyber and information security. This document also provides examples and suggests strategies for eliminating or mitigating the associated risk.

Purpose

Despite the sophistication and complicated methodologies employed, ML systems can introduce risks to safety by learning incorrectly, making wrong inferences, and then recommending or initiating actions that can lead to harm. The amplification of errors in an AI system has the potential to create large scale harm to patients. Sometimes these systems detect correlations in data sets instead of causations, which can lead to incorrect conclusions. All medical devices come with inherent risks. Manufacturers are required to demonstrate that their medical devices do not pose unacceptable risks, and that the benefits of their intended use outweigh the overall residual risk. ISO 14971:2019, Medical devices — Application of risk management to medical devices, details how manufacturers assess and mitigate potential risks in order to protect the health and safety of patients as well as data and system security. Additionally, IEC 80001-1:2010, Application of risk management for IT-networks incorporating medical devices — Part 1: Roles, responsibilities and activities addresses risks for IT networks incorporating medical devices and IEC/TR 80002-1:2009, Medical device software — Part 1: Guidance on the application of ISO 14971 to medical device software addresses software internal to the medical device that might support AI. These standards, and associated supporting guidance documents, provide the basis for risk management and the lifecycle process for all software that is regulated under medical device legislation.

One of the key findings of the 2020 AAMI/BSI whitepaper was the need to develop “risk management guidance to assist in applying ISO 14971 to AI as a medical technology.” ISO 14971, developed by ISO/TC 210 – IEC/SC 62A/JWG 1, provides a process for managing the risk associated with medical devices. It has been recognized by medical device regulators and adopted as a national standard in countries across the world. This proposal does not provide a new risk management process, nor does it expand the requirements of ISO 14971. Rather, it provides guidance to assist those who are applying ISO 14971 to regulated AI medical technologies.