FHIRTrustBench: A Benchmark for Interoperability-Driven Clinical AI Readiness and Trustworthiness

Existing evaluations of healthcare AI often treat interoperability as a technical infrastructure issue rather than a factor that directly influences the safety and reliability of clinical AI systems. Yet the quality of Fast Healthcare Interoperability Resources (FHIR) implementation affects whether AI models can operate accurately, fairly, securely, and effectively in real clinical settings. We present FHIRTrustBench, a benchmark for assessing the readiness of FHIR-based clinical AI systems across five complementary dimensions: FHIR implementation quality, AI validation, clinical workflow integration, trustworthiness assessment, and governance readiness. Each dimension is mapped to a distinct category of downstream deployment failure risk. We applied FHIRTrustBench to a corpus of 10 representative sources spanning interoperability standards, implementation studies, electronic health record integration research, healthcare large language model research, and governance frameworks. Each source was scored individually and traceably against the five-dimension rubric. FHIR Specificity achieved the highest dimension mean at 1.3 out of 2.0, while AI Validation received the lowest at 0.3. Even category-leading sources that scored a maximum 2.0 on FHIR Specificity scored 0 on AI Validation. Prospective external validation was reported in no source, and Governance Readiness remained at or below 1.0 across every category. We further identify five interoperability-related AI failure pathways, spanning data integrity, semantic consistency, security, clinical workflow, and generative AI grounding, and propose a deployment lifecycle framework and reporting checklist that translate benchmark scores into deployment-readiness decisions for developers, healthcare organizations, and regulators. FHIRTrustBench provides a practical and reproducible basis for assessing FHIR-enabled clinical AI before deployment and can evolve as interoperability standards and clinical evidence mature.