Enhancing Antibody Language Models with Structural Information
Barton, J.; Galson, J. D.; Leem, J.
Show abstract
The central tenet of molecular biology is that a proteins amino acid sequence determines its three-dimensional structure, and thus its function. However, proteins with similar sequences do not always fold into the same shape, and vice-versa, dissimilar sequences can adopt similar folds. In this work, we explore antibodies, a class of proteins in the immune system, whose local shapes are highly unpredictable, even with small variations in their sequence. Inspired by the CLIP method [1], we propose a multimodal contrastive learning approach, contrastive sequence-structure pre-training (CSSP), which amalgamates the representations of antibody sequences and structures in a mutual latent space. Integrating structural information leads both antibody and protein language models to show better correspondence with structural similarity and improves accuracy and data efficiency in downstream binding prediction tasks. We provide an optimised CSSP-trained model, AntiBERTa2-CSSP, for non-commercial use at https://huggingface.co/alchemab.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Learning Context-aware Structural Representations to Predict Antigen and Antibody Binding Interfaces 98%
- Paragraph - Antibody paratope prediction using Graph Neural Networks with minimal feature vectors 97%
- Beyond the Leaderboard: Leveraging Predictive Modeling for Protein-Ligand Insights and Discovery 97%
Similar papers in this journal
- Paraplume: A fast and accurate paratope prediction method provides insights into repertoire-scale binding dynamics 97%
- Nucleotide context models outperform protein language models for predicting antibody affinity maturation 96%
- Controllable Protein Design via Autoregressive Direct Coupling Analysis Conditioned on Principal Components 95%
Similar papers in this journal
- Humatch - fast, gene-specific joint humanisation of antibody heavy and light chains 95%
- BioPhi: A platform for antibody design, humanization and humanness evaluation based on natural antibody repertoires and deep learning 95%
- Towards generalizable prediction of antibody thermostability using machine learning on sequence and structure features 95%
Similar papers in this journal
- CASTER-DTA: Equivariant Graph Neural Networks for Predicting Drug-Target Affinity 96%
- PLMFit : Benchmarking Transfer Learning with Protein Language Models for Protein Engineering 95%
- Scalable embedding fusion with protein language models: insights from benchmarking text-integrated representations 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.