When Geometric Deep Learning Meets Pretrained Protein Language Models
Wu, F.; Radev, D.; Xu, J.
Show abstract
Geometric deep learning has recently achieved great success in non-Euclidean domains, and learning on 3D structures of large biomolecules is emerging as a distinct research area. However, its efficacy is largely constrained due to the limited quantity of structural data. Meanwhile, protein language models trained on substantial 1D sequences have shown burgeoning capabilities with scale in a broad range of applications. Nevertheless, no preceding studies consider combining these different protein modalities to promote the representation power of geometric neural networks. To address this gap, we make the foremost step to integrate the knowledge learned by well-trained protein language models into several state-of-the-art geometric networks. Experiments are evaluated on a variety of protein representation learning benchmarks, including protein-protein interface prediction, model quality assessment, protein-protein rigid-body docking, and binding affinity prediction, leading to an overall improvement of 20% over baselines and the new state-of-the-art performance. Strong evidence indicates that the incorporation of protein language models knowledge enhances geometric networks capacity by a significant margin and can be generalized to complex tasks.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Pair-EGRET: enhancing the prediction of protein-proteininteraction sites through graph attention networks and protein language models 97%
- Topology-Driven Negative Sampling Enhances Generalizability in Protein-Protein Interaction Prediction 97%
- FAPM: Functional Annotation of Proteins using Multi-Modal Models Beyond Structural Modeling 97%
Similar papers in this journal
- Predicting Affinity Through Homology (PATH): Interpretable Binding Affinity Prediction with Persistent Homology 96%
- BAGEL: Protein Engineering via Exploration of an Energy Landscape 95%
- Computational design of novel Cas9 PAM-interacting domains using evolution-based modelling and structural quality assessment 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.