Evaluating Pre-trained 3rd Party AI Models in Medical Device Software Development for Responsible and Ethical Use

It is well understood that new standards are necessary to ensure the use of Artificial Intelligence-enabled Medical Devices (AIeMD) are adequately controlled. CEN-CENELEC (European focus) and IEC (Global focus) are developing such standards. Yet the use of pre-trained AI models, available as Software of Unknown Provenance (SOUP) from 3rd party suppliers, remain excluded from the standards under development. This paper aims to demonstrate the need for standardisation of the qualification process for these models beyond that understood by existing standards such as ISO 13485 and IEC 62304. This is necessary in order to achieve responsible and ethical AI for use in healthcare. This paper demonstrates the need for guidance by setting out emerging use cases as well as the risks and potential risk mitigations available when using pre-trained models as SOUP in the Software Development Lifecycle of AIeMD. It goes further to outline the necessary characteristics that should be standardsed. Evaluation questions are provided and demonstrate salient points necessary for responsible and ethical AI.