Codon language embeddings provide strong signals for protein engineering
Outeiral, C.; Deane, C.
Show abstract
Protein representations from deep language models have yielded state-of-the-art performance across many tasks in computational protein engineering. In recent years, progress has primarily focused on parameter count, with recent models capacities surpassing the size of the very datasets they were trained on. Here, we propose an alternative direction. We show that large language models trained on codons, instead of amino acid sequences, provide high-quality representations that outperform comparable state-of-the-art models across a variety of tasks. In some tasks, like species recognition, prediction of protein and transcript abundance, or melting point estimation, we show that a language model trained on codons outperforms every other published protein language model, including some that contain over 50 times more parameters. These results suggest that, in addition to commonly studied scale and model complexity, the information content of biological data provides an orthogonal direction to improve the power of machine learning in biology.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Discovering molecular features of intrinsically disordered regions by using evolution for contrastive learning 98%
- Generating functional protein variants with variational autoencoders 96%
- Paraplume: A fast and accurate paratope prediction method provides insights into repertoire-scale binding dynamics 95%
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.