Biased sampling confounds machine learning prediction of antimicrobial resistance
Yu, Y.; Wheeler, N. E.; Barquist, L.
Show abstract
Antimicrobial resistance (AMR) poses a growing threat to human health. Increasingly, genome sequencing is being applied for the surveillance of bacterial pathogens, producing a wealth of data to train machine learning (ML) applications to predict AMR and identify resistance determinants. However, bacterial populations are highly structured and sampling is biased towards human disease isolates, meaning samples and derived features are not independent. This is rarely considered in applications of ML to AMR. Here, we demonstrate the confounding effects of sample structure by analyzing over 24,000 whole genome sequences and AMR phenotypes from five diverse pathogens, using pathological training data where resistance is confounded with phylogeny. We show resulting ML models perform poorly, and increasing the training sample size fails to rescue performance. A comprehensive analysis of 6,740 models identifies species- and drug-specific effects on model accuracy. We provide concrete recommendations for evaluating future ML approaches to AMR.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- A convolutional neural network highlights mutations relevant to antimicrobial resistance in Mycobacterium tuberculosis 95%
- Using big sequencing data to identify chronic SARS-Coronavirus-2 infections 94%
- Quantitative measurement of antibiotic resistance in Mycobacterium tuberculosis reveals genetic determinants of resistance and susceptibility in a target gene approach 94%
Similar papers in this journal
- Metapopulation ecology links antibiotic resistance, consumption and patient transfers in a network of hospital wards 92%
- Host-parasite coevolution promotes innovation through deformations in fitness landscapes 92%
- Interpreting roles of mutations associated with the emergence of S. aureus USA300 strains using transcriptional regulatory network reconstruction 92%
Similar papers in this journal
- Integration of multi-modal measurements identifies critical mechanisms of tuberculosis drug action 94%
- Learning the Language of Phylogeny with MSA Transformer 93%
- Markov Field network integration of multi-modal data predicts effects of immune system perturbations on intravenous BCG vaccination in macaques 92%
Similar papers in this journal
- Horizontal gene transfer rate is not the primary determinant of observed antibiotic resistance frequencies in Streptococcus pneumoniae 93%
- Emergence and global spread of Listeria monocytogenes main clinical clonal complex 93%
- Genome-scale phylogeny and contrasting modes of genome evolution in the fungal phylum Ascomycota 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.