Melchior: A Hybrid Mamba-Transformer RNA Basecaller
Litman, E.
Show abstract
AO_SCPLOWBSTRACTC_SCPLOWSequence transduction from raw nanopore signals is notoriously difficult because the signal level does not naturally correspond to a single base, but rather many adjacent nucleotides. Thus, we introduce Melchior, an RNA basecaller that uses a hybrid Mamba-Transformer backbone to achieve global visual context at a lower computational complexity. This is in contrast to temporal convolutions, which collate features by fusing both spatial and channel features in the local receptive field. Melchior is also able to exploit the full complementarity between local and global features, unlike Vision Transformers, which have empirically been observed to ignore local features. Augmenting a selective structured state-space sequence model with self-attention unlocks unprecedented performance gains, particularly in homopolymer regions, by modeling fine-grained details in both short and long-range spatial dependencies.
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Graph Contrastive Learning of Subcellular-resolution Spatial Transcriptomics Improves Cell Type Annotation and Reveals Critical Molecular Pathways 95%
- CRISPR-DIPOFF: An Interpretable Deep LearningApproach for CRISPR Cas-9 Off-Target Prediction 94%
- Spatial Transcriptomics Prediction from Histology jointly through Transformer and Graph Neural Networks 94%
Similar papers in this journal
- CellFM: a large-scale foundation model pre-trained on transcriptomics of 100 million human cells 95%
- Benchmarking DNA Foundation Models for Genomic and Genetic Tasks 95%
- Hi-C-LSTM: Learning representations of chromatin contacts using a recurrent neural network identifies genomic drivers of conformation 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.