Considerations for performance metrics of metagenomic next generation sequencing analyses
Kralj, J. G.; Servetas, S. L.; Forry, S. P.; Jackson, S. A.
Show abstract
Evaluating the performance of metagenomics analyses has proven a challenge, due in part to limited ground-truth standards, broad application space, and numerous evaluation methods and metrics. Application of traditional clinical performance metrics (i.e. sensitivity, specificity, etc.) using taxonomic classifiers do not fit the "one-bug-one-test" paradigm. Ultimately, users need methods that evaluate fitness-for-purpose and identify their analyses strengths and weaknesses. Within a defined cohort, reporting performance metrics by taxon, rather than by sample, will clarify this evaluation. An estimated limit of detection, positive and negative control samples, and true positive and negative true results are necessary criteria for all investigated taxa. Use of summary metrics should be restricted to comparing results of similar cohorts and data, and should employ harmonic means and continuous products for each performance metric rather than arithmetic mean. Such consideration will ensure meaningful comparisons and evaluation of fitness-for-purpose.
Matching journals
The top 10 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- GAMBIT (Genomic Approximation Method for Bacterial Identification and Tracking): A methodology to rapidly leverage whole genome sequencing of bacterial isolates for clinical identification 94%
- centriflaken: an automated data analysis pipeline for assembly and in silico analyses of foodborne pathogens from metagenomic samples 92%
- Sample pooling methods for efficient pathogen screening: Practical implications 91%
Similar papers in this journal
- Analytical Assessment of Metagenomic Workflows for Pathogen Detection with NIST RM 8376 and Two Sample Matrices 94%
- Machine-learning based detection of adventitious microbes in T-cell therapy cultures using long read sequencing 94%
- Evaluation of the Ultima Genomics UG 100 sequencer for low-cost, high-sensitivity metagenomic pathogen detection from cerebrospinal fluid 93%
Similar papers in this journal
- Analytical and Clinical Comparison of Three Nucleic Acid Amplification Tests for SARS-CoV-2 Detection 92%
- Estimating the false positive rate of highly automated SARS-CoV-2 nucleic acid amplification testing 92%
- Performance characteristics of a high throughput automated transcription mediated amplification test for SARS-CoV-2 detection 91%
Similar papers in this journal
- Comprehensive benchmarking of metagenomic classification tools for long-read sequencing data 91%
- Benchmarking workflows to assess performance and suitability of germline variant calling pipelines in clinical diagnostic assays. 91%
- Natrix: A Snakemake-based workflow for processing, clustering, and taxonomically assigning amplicon sequencing reads 90%
Similar papers in this journal
- Development and validation of an HPLC method to quantify 2-Keto-3-deoxy-gluconate (KDG), a major metabolite in pectin and alginate degradation pathways 88%
- Multi-Wavelength Analytical Ultracentrifugation of Biopolymer Mixtures and Interactions 87%
- Parameter estimation and identifiability analysis for a bivalent analyte model of monoclonal antibody-antigen binding 87%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.