What does it take to learn the rules of RNA base pairing? A lot less than you may think
Pratap, J. S.; Krueger, R. K.; Rivas, E.
Show abstract
Amidst the fast-developing trend of RNA large language models with millions of parameters, we asked what would be the minimum required to rediscover the rules of RNA canonical base pairing, mainly the Watson-Crick-Franklin A:U, G:C and the wobble G:U base pairs (the secondary structure). Here, we conclude that it does not require much at all. It does not require knowing secondary structures; it does not require aligning the sequences; and it does not require many parameters. We selected a probabilistic model of palindromes (a stochastic context-free grammar or SCFG) with a total of just 21 parameters. Using standard deep learning techniques, we estimate its parameters by implementing the generative process in an automatic differentiation (autodiff) framework and applying stochastic gradient descent (SGD). We define and minimize a loss function that does not use any structural or alignment information. Trained on as few as fifty RNA sequences, the rules of RNA base pairing emerge after only a few iterations of SGD. Crucially, the sole inputs are RNA sequences. When optimizing for sequences corresponding to structured RNAs, SGD also yields the rules of RNA base-pair aggregation into helices. Trained on shuffled sequences, the system optimizes by avoiding base pairing altogether. Trained on messenger RNAs, it reveals interactions that are different from those of structural RNAs, and specific to each mRNA. Our results show that the emergence of canonical base-pairing can be attributed to sequence-level signals that are robust and detectable even without labeled structures or alignments, and with very few parameters. Autodiff algorithms for probabilistic models, such as, but not restricted to SCFGs, have significant potential as they allow these models to be incorporated into end-to-end RNA deep learning methods for discerning transcripts of different functionalities. Availabilityrivaslab.org, https://github.com/EddyRivasLab/R-scape/tree/master/python/d-SCFG
Matching journals
The top 4 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Informative RNA-base embedding for functional RNA structural alignment and clustering by deep representation learning 96%
- Tailored machine learning models for functional RNA detection in genome-wide screens 95%
- Analyzing Genomic Data Using Tensor-Based Orthogonal Polynomials with Application to Synthetic RNAs 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.