scMARK an 'MNIST' like benchmark to evaluate and optimize models for unifying scRNA data
Singh, S.; Mendonca, D.; Focsa, O.; Diaz-Mejia, J. J.; Cooper, S.
Show abstract
Todays single-cell RNA analysis tools provide enormous value in enabling researchers to make sense of large single-cell RNA (scRNA) studies, yet their ability to integrate different studies at scale remains untested. Here we present a novel benchmark dataset (scMARK), that consists of 100,000 cells over 10 studies and can test how well models unify data from different scRNA studies. We also introduce a two-step framework that uses supervised models, to evaluate how well unsupervised models integrate scRNA data from the 10 studies. Using this framework, we show that the Variational Autoencoder, scVI, represents the only tool tested that can integrate scRNA studies at scale. Overall, this work paves the way to creating large scRNA atlases and off-the-shelf analysis tools.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- scValue: value-based subsampling of large-scale single-cell transcriptomic data for machine and deep learning tasks 96%
- scDeepInsight: a supervised cell-type identification method for scRNA-seq data with deep learning 96%
- A comprehensive comparison on cell type composition inference for spatial transcriptomics data 96%
Similar papers in this journal
- A systematic evaluation of highly variable gene selection methods for single-cell RNA-sequencing 96%
- Benchmarking algorithms for joint integration of unpaired and paired single-cell RNA-seq and ATAC-seq data 96%
- Beyond benchmarking: towards predictive models of dataset-specific single-cell RNA-seq pipeline performance 96%
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.