DGRNA: a long-context RNA foundation model with bidirectional attention Mamba2
Yuan, Y.; Chen, Q.; Pan, X.
Show abstract
Ribonucleic acid (RNA) is an important biomolecule with diverse functions i.e. genetic information transfer, regulation of gene expression and cellular functions. In recent years, the rapid development of sequencing technology has significantly enhanced our understanding of RNA biology and advanced RNA-based therapies, resulting in a huge volume of RNA data. Data-driven methods, particularly unsupervised large language models, have been used to automatically hidden semantic information from these RNA data. Current RNA large language models are primarily based on Transformer architecture, which cannot efficiently process long RNA sequences, while the Mamba architecture can effectively alleviate the quadratic complexity associated with Transformers. In this study, we propose a large foundational model DGRNA based on the bidirectional Mamba trained on 100 million RNA sequences, which has demonstrated exceptional performance across six RNA downstream tasks compared to existing RNA language models.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- UTRGAN: Learning to Generate 5' UTR Sequences for Optimized Translation Efficiency and Gene Expression 96%
- LinAliFold and CentroidLinAliFold: Fast RNA consensus secondary structure prediction for aligned sequences using beam search methods 96%
- Prediction of RNA-protein interactions using a nucleotide language model 95%
Similar papers in this journal
Similar papers in this journal
- sincFold: end-to-end learning of short- and long-range interactions in RNA secondary structure 97%
- A Reproducibility Analysis-based Statistical Framework for Residue-Residue Evolutionary Coupling Detection 95%
- DeepLncLoc: a deep learning framework for long non-coding RNA subcellular localization prediction based on subsequence embedding 95%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.