Protocol for the AutoRayValid-RBfracture Study: Evaluating the efficacy of an AI fracture detection system

BackgroundRapidly diagnosing fractures in appendicular skeletons is vital in the ED, where junior physicians often interpret initial radiographs. However, missed fractures remain a concern, prompting AI-assisted detection exploration. Yet, existing studies lack clinical context. We propose a multi-center retrospective study evaluating the AI aid RBfracture v.1, aiming to assess AIs impact on diagnostic thinking by analyzing consecutive cases with clinical data, providing insights into fracture detection and clinical decision-making. ObjectivesTo provide new insights on the potential value of AI tools across borders and different healthcare systems. We will evaluate the performance of the AI aid to detect fractures on conventional x-ray images and how its use could affect handling of these cases in a healthcare setting. In order to explore if the use of a trained and certified AI tool on clinical data exposes new challenges, a daily practice clinical scenario will be approached by minimising selection criteria and using consecutive cases. A multicenter, retrospective, diagnostic accuracy cross-sectional design incorporates clinical context. MethodsThe multicenter study spans three European sites without onsite hardware. AI system RBfracture v.1 maintains consistent sensitivity and specificity thresholds. Eligibility involves age [≥]21 with x-ray indications for appendicular fractures. Exclusions include casts, follow-up x-rays, nearby hardware. AI aids retrospective fracture detection. Reader sessions include radiology and emergency care residents and trainees reading with and without AI. Fractures are marked, rated, with expert-established reference standards. DataSequential patient studies at three sites yield 500 cases per site. Data includes anatomy, referral notes, radiology reports, and radiographic images. Expert readers use annotations, clinical context for standards. Statistical methods include dichotomized confidence ratings, sensitivity, specificity calculations, site-based analysis and subgroup considerations. Reference StandardTwo experienced readers annotate fractures; if their annotations overlap by 25% or more, the common area is the reference. Discrepancies are resolved by a local expert. Individual fractures are labelled.