Cross-database validation reveals distinct layers of transportability in ICU delirium prediction
Ni, S.; Sato, K.
Show abstract
External validation of clinical AI emphasizes discrimination, although deployment requires the endpoint, probability estimates and operating policy to transport. Here we show that these layers diverged in retrospective bidirectional evaluation of five model families across eICU and MIMIC-IV. Coarse-label AUROC fell from 0.87-0.92 internally to 0.66-0.83 during source-only transfer. For assessment-conditioned repeated monitoring of persistence or recurrence, external AUROC reached 0.76-0.94, but removing assessment history reduced it by 0.16-0.32; broader features did not help consistently. Transported scores concentrated future-positive ICU stays 2.4-6.9-fold in the top risk decile. Development-selected cutoffs alerted 0.3-2.0% of prediction rows and captured 9.2-11.0% of future-positive rows; after deduplication, 4.9-12.2% of stays were alerted, capturing 43.9-49.4% of future-positive stays. Thus, ranking can persist while probability and policy transport remain site dependent. Layered validation is a prerequisite for prospective evaluation, not evidence of clinical benefit.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- An Immune Cell Atlas Reveals Dynamic COVID-19 Specific Neutrophil Programming Amenable to Dexamethasone Therapy 91%
- Zero-shot drug repurposing with geometric deep learning and clinician centered design 91%
- Machine learning guided signal enrichment for ultrasensitive plasma tumor burden monitoring 91%
Similar papers in this journal
- Development and assessment of a machine learning tool for predicting emergency admission in Scotland 93%
- Federated Target Trial Emulation using Distributed Observational Data for Treatment Effect Estimation 92%
- High-Sensitivity Pan-Cancer AI Assessment of Lymph Node Metastasis via Uncertainty Quantification 91%
Similar papers in this journal
- AI-guided discovery of the invariant host response to viral pandemics 90%
- Transformer-based deep learning model for the diagnosis of suspected lung cancer in primary care based on electronic health record data 90%
- Integrative deep immune profiling of the elderly reveals systems-level signatures of aging, sex, smoking, and clinical traits 90%
Similar papers in this journal
- Integrated immunovirological profiling validates plasma SARS-CoV-2 RNA as an early predictor of COVID-19 mortality 91%
- Charting the transcriptomic landscape of primary and metastatic cancers in relation to their origin and target normal tissues 90%
- Biologically relevant integration of transcriptomics profiles from cancer cell lines, patient-derived xenografts and clinical tumors using deep learning 90%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.