The Coalition for Health AI (CHAI) has released draft frameworks for certifying independent assurance labs and a nutrition label for health artificial intelligence (AI) technologies.
These labs will test health AI models using CHAI Model Cards, which function as ingredient and nutrition labels.
The CHAI certification process and model card design will be available by the end of April 2025. This timeline includes review and feedback from CHAI members, partners, and the public.
CHAI CEO Brian Anderson said: “This has been a pivotal year for CHAI on our journey to help enable trusted independent assurance of AI solutions with local monitoring and validation.
“We are thrilled with the progress of our CHAI workgroups which represent a diversity of perspectives and expertise across the health ecosystem.
“These frameworks for certification and basic transparency are building blocks of responsible health AI.
“They will help to streamline the path for AI innovation, build trust with patients and clinicians, and position health systems and solution innovators ahead of emerging state and federal regulations.”
The draft CHAI certification programme framework was developed in collaboration with the ANSI National Accreditation Board and various quality assurance labs, following ISO 17025 standards for testing and calibration laboratories worldwide.
Key requirements include mandatory disclosure of conflicts of interest between assurance labs and model developers, along with protections for data and intellectual property.
Additionally, the certification programme incorporates data quality and integrity standards based on US Food and Drug Administration (FDA) guidance for real-world data.
It also utilises CHAI testing and evaluation metrics from multiple working groups and aligns with the National Academy of Medicine’s AI Code of Conduct.
The draft CHAI Model Card provides a standardised template to enhance transparency in evaluating AI solution performance and safety. It includes essential information such as the developer’s identity, intended uses, target patient populations, AI model type, data types, key performance metrics, and others.
Additionally, the model card will serve as a foundational tool for reviewing AI models during the procurement process and will assist EHR vendors in complying with the Office of the National Coordinator for Health Information Technology (ONC) Health IT Certification Program.
The coalition plans to engage stakeholders from the healthcare ecosystem to gather additional feedback.
