By Prof Kostas Marias Hellenic Mediterranean University, Head Computational BioΜedicine Lab – FORTH
Enabling AI Models Traceability through ProCAncer-I AI Model Passport
Artificial intelligence (AI) has brought great economic and social prospects to human society with a great impact on several areas. Specifically, in the health domain, recent advances in AI have proven to match or even surpass humans in several tasks, e.g., radiological tasks. However, to realize the full potential of AI in medical applications, one should take into consideration the potential risks and ethical implications of health AI solutions. Stakeholders have pinpointed the trustworthiness issue of such systems, as these are perceived as black boxes, which are hard to fathom and explain. For achieving the vision of Trustworthy AI a set of key requirements should be met. One of those requirements is transparency, a complex construct that according to the High-Level Expert Group on AI consists of three components, i.e., traceability, explainability, and open communication about the limitations of the AI system.
In relation to the ProCAncer-I project, our initial emphasis was on traceability, which refers to the mandate to document the whole development process and to track the functioning of an AI model or an AI-based system used to support further analysis and interpretation. Since the variability of AI is high, this documentation should be complete and detailed. More specifically, the data sets, the processes, the reference clinical gold standards, and any human-made decision that takes part in an AI system should be documented to the best possible standard in order to achieve the realization of traceability. Both the scientific community and the industry have already recognized the necessity of a data model and the relevant tools for capturing the pertinent information. However, existing approaches focus only on some of the steps of the AI model development and
usage ignoring significant parts of the whole process.
Within the ProCAncer-I project we introduce for the first time the notion of the AI passport for tracking and documenting all steps involved in the AI model training and usage starting from data collection, data curation, training data preparation, model training, and validation as well as model monitoring in production. The concept of the AI model passport enables end-to-end traceability of the whole AI lifecycle. This involves documenting the entire development process of an AI model/system, including the data utilized during training and validation, the actors involved, and
the processing and fine-tuning procedures implemented. This approach is particularly critical in high-stakes sectors such as healthcare, where delivering a traceable AI solution is an essential prerequisite. By ensuring complete traceability, the AI solution can be reliably integrated into clinical practice, making it a vital and safe solution.
Figure 1 Example AI-Passport for a Segmentation AI Model
.
