Transforming the Language of Life: Transformer Neural Networks for Protein Prediction Tasks
Nambiar, A.; Heflin, M. E.; Liu, S.; Maslov, S.; Hopkins, M.; Ritz, A.
Show abstract
AO_SCPLOWBSTRACTC_SCPLOWThe scientific community is rapidly generating protein sequence information, but only a fraction of these proteins can be experimentally characterized. While promising deep learning approaches for protein prediction tasks have emerged, they have computational limitations or are designed to solve a specific task. We present a Transformer neural network that pre-trains task-agnostic sequence representations. This model is fine-tuned to solve two different protein prediction tasks: protein family classification and protein interaction prediction. Our method is comparable to existing state-of-the art approaches for protein family classification, while being much more general than other architectures. Further, our method outperforms all other approaches for protein interaction prediction. These results offer a promising framework for fine-tuning the pre-trained sequence representations for other protein prediction tasks.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- FAPM: Functional Annotation of Proteins using Multi-Modal Models Beyond Structural Modeling 98%
- Pair-EGRET: enhancing the prediction of protein-proteininteraction sites through graph attention networks and protein language models 98%
- ProteinBERT: A universal deep-learning model of protein sequence and function 97%
Similar papers in this journal
- ICoN: Integration using Co-attention across Biological Networks 97%
- Improving protein function prediction by learning and integrating representations of protein sequences and function labels 97%
- KSMoFinder - Knowledge graph embedding of proteins and motifs for predicting kinases of human phosphosites 97%
Similar papers in this journal
Similar papers in this journal
- Struct2Graph: A graph attention network for structure based predictions of protein-protein interactions 97%
- Multi-Head Attention-based U-Nets for Predicting Protein Domain Boundaries Using 1D Sequence Features and 2D Distance Maps 96%
- COMIC: Explainable Drug Repurposing via Contrastive Masking for Interpretable Connections 95%
Similar papers in this journal
- Paying Attention to Attention: High Attention Sites as Indicators of Protein Family and Function in Language Models 96%
- PandoGen: Generating complete instances of future SARS-CoV-2 sequences using Deep Learning 96%
- Highly Accurate Cancer Phenotype Prediction with AKLIMATE, a Stacked Kernel Learner Integrating Multimodal Genomic Data and Pathway Knowledge 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.