Assessment of AUC and fold change for precision medicine
Climer, S.; Smith, K.
Show abstract
Precision medicine is advancing patient care for complex human diseases. Discovery of biomarkers to diagnose specific subtypes within a heterogeneous diseased population is a key step towards realizing the benefits of precision medicine. However, popular statistical methods for evaluating candidate biomarkers - fold change (FC) and area under the receiver operating characteristic curve (AUC) - were designed for homogeneous data. Herein, we evaluate the performance of these metrics in heterogeneous populations. Using simulated biomarkers that are nearly ideal for distinguishing subgroups of various proportions of the diseased population, we observe that AUC misses all up to subset size of 50% and FC misses all biomarkers entirely. We introduce a simple new measure to address this shortfall and run a series of trials comprised of simulated and biological data to demonstrate its utility for evaluating biomarkers associated with disease subtypes.
Matching journals
The top 8 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Selecting the most important self-assessed features for predicting conversion to Mild Cognitive Impairment with Random Forest and Permutation-based methods 93%
- Co-localized SNPs Affecting the Expression of Taste Perception Genes are linked to Alzheimer's Disease 92%
- Repurposing Non-pharmacological Interventions for Alzheimer’s Diseases through Link Prediction on Biomedical Literature 91%
Similar papers in this journal
- Characterizing subgroup performance of probabilistic phenotype algorithms within older adults: A case study for dementia, mild cognitive impairment, and Alzheimer’s and Parkinson’s diseases 92%
- Clinical interpretation of machine learning models for prediction of diabetic complications using electronic health records 88%
- Large-Scale Deep Learning for Metastasis Detection in Pathology Reports 88%
Similar papers in this journal
- CharMark: A Markov Approach to Linguistic Biomarkers in Dementia 90%
- Retinal Vascular Measures from Diabetes Retinal Screening Photographs and Risk of Incident Dementia in Type 2 Diabetes: A GoDARTS Study 89%
- Using Machine Learning to Predict Mortality for COVID-19 Patients on Day Zero in the ICU 89%
Similar papers in this journal
- Interpretable deep learning approach for extracting cognitive features from hand-drawn images of intersecting pentagons in older adults 92%
- Machine Learning Prediction of Incidence of Alzheimers Disease Using Large-Scale Administrative Health Data 92%
- A scoping review of remote and unsupervised digital cognitive assessments in preclinical Alzheimer’s disease 92%
Similar papers in this journal
- Using Machine Learning and Electronic Health Record (EHR) Data for the Early Prediction of Alzheimer’s Disease and Related Dementias 94%
- Identifying Proteomic Prognostic Markers for Alzheimer’s Disease with Survival Machine Learning: the Framingham Heart Study 91%
- Real-world datasets for the International Registry for Alzheimer’s Disease and Other Dementias (InRAD) and other registries: an international consensus 90%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.