Back

Standardised evaluation and monitoring of site-specific AI performance with physical CT phantoms

Genske, U.; Laudani, A.; Yan, L.; Peng, Y.; Boening, G.; Ulas, S. T.; Wagner, M. P.; Diekhoff, T.; Hamm, B.; Jahnke, P.

2026-07-02 radiology and imaging
10.64898/2026.07.01.26357033 medRxiv
Show abstract

Artificial intelligence (AI) applications in computed tomography (CT) imaging require objective and continuous testing, yet standardised methods for this purpose have not been established. Here, we present a framework using physical phantoms for standardised testing and monitoring of AI, demonstrated in liver lesion detection. We begin by designing phantoms tailored to the anatomical input domain expected by AI algorithms, and then systematically assess how AI performance is affected by variations in scanner technology and operation across two clinical CT systems. Next, we perform longitudinal monitoring, yielding consistent results over fifteen months on both systems. Finally, we validate clinical relevance by demonstrating that AI models trained on phantom data generalize effectively to patients and exhibit no evidence of phantom-specific adaptation. Our findings show that anatomically realistic phantoms enable standardised, site-specific testing and monitoring of AI, providing a proactive method for local and cross-institutional quality assurance.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
Nature Communications
○ 5641 papers in training set
Top 12%
15.1%
Similar papers in this journal
2
Medical Physics
○ 14 papers in training set
Top 0.1%
15.1%
Similar papers in this journal
3
Scientific Reports
○ 3612 papers in training set
Top 4%
10.6%
Similar papers in this journal
4
Physics in Medicine & Biology
○ 18 papers in training set
Top 0.1%
6.7%
Similar papers in this journal
5
European Radiology
○ 15 papers in training set
Top 0.1%
5.2%
Similar papers in this journal
50% of probability mass above
6
Journal of Medical Imaging
● 11 papers in training set
Top 0.1%
4.3%
7
Scientific Data
○ 209 papers in training set
Top 0.8%
3.2%
8
eBioMedicine
○ 183 papers in training set
Top 2%
2.4%
9
Photoacoustics
○ 12 papers in training set
Top 0.1%
2.4%
10
Communications Biology
○ 993 papers in training set
Top 10%
2.1%
11
Frontiers in Oncology
○ 103 papers in training set
Top 2%
2.1%
12
NeuroImage
○ 903 papers in training set
Top 5%
1.7%
13
Science Advances
● 1243 papers in training set
Top 19%
1.7%
14
npj Precision Oncology
○ 53 papers in training set
Top 0.9%
1.7%
15
Nature Machine Intelligence
○ 70 papers in training set
Top 2%
1.5%
16
PLOS ONE
● 5266 papers in training set
Top 50%
1.3%
17
Magnetic Resonance in Medicine
○ 85 papers in training set
Top 0.5%
1.1%
18
Clinical and Translational Radiation Oncology
○ 10 papers in training set
Top 0.2%
1.1%
19
GigaScience
◐ 212 papers in training set
Top 4%
0.9%
20
Annals of Biomedical Engineering
○ 37 papers in training set
Top 1.0%
0.9%
21
JCO Clinical Cancer Informatics
● 22 papers in training set
Top 0.7%
0.8%
22
Frontiers in Medicine
○ 120 papers in training set
Top 4%
0.8%
23
Medical Image Analysis
○ 35 papers in training set
Top 0.7%
0.8%
24
eLife
● 5828 papers in training set
Top 66%
0.8%
25
Aperture Neuro
● 20 papers in training set
Top 0.5%
0.8%
26
Biology Methods and Protocols
◐ 61 papers in training set
Top 2%
0.8%
27
SLAS Technology
○ 14 papers in training set
Top 0.2%
0.8%
28
Nature Medicine
○ 125 papers in training set
Top 4%
0.6%
29
The Lancet Digital Health
○ 25 papers in training set
Top 1%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.