On the assessment of deep-learning based super-resolution in small datasets of human brain MRI scans

Deep-learning based super-resolution has shown promise for enhancing the spatial resolution of brain magnetic resonance images, which may help visualize small anatomical structures more clearly. However, when only limited training data are available, it remains uncertain which model assessment method provides the most reliable estimate of out-of-sample performance. In this study, three widely used assessment strategies (three-way holdout, k-fold cross-validation, and nested cross-validation) were compared for evaluating the performance of such models in small datasets. Across 30 iterations, we randomly selected subsets of 20 T2-weighted images from the 1,113 scans of the Human Connectome Project. Each subset was used to train a model and estimate performance using the three methods. The ground truth error was computed from the remaining images. The assessment error is the difference between the estimated error and the ground truth error. The median assessment errors were 0.11,- 0.13, - 0.32 for three-way holdout, k-fold cross-validation, and nested cross-validation, respectively, with the cross-validation methods showing considerably smaller dispersions. Nested cross-validation selected fewer epochs, indicating more conservative model selection, but required substantially greater computational time, over three times longer than three-way holdout and more than twenty times longer than k-fold cross-validation. Our findings suggest that k-fold cross-validation offers the most favourable balance between accuracy, stability, and computational feasibility in small datasets. Further research is needed to determine how model complexity, dataset size, and the number of cross-validation folds influence assessment accuracy.