Multi-Scale Protein Language Model for Unified Molecular Modeling
Zheng, K.; Long, S.; Lu, T.; Yang, J.; Dai, X.; Zhang, M.; Nie, Z.; Ma, W.-Y.; Zhou, H.
Show abstract
Protein language models have demonstrated significant potential in the field of protein engineering. However, current protein language models primarily operate at the residue scale, which limits their ability to provide information at the atom level. This limitation prevents us from fully exploiting the capabilities of protein language models for applications involving both proteins and small molecules. In this paper, we propose ESM-AA (ESM All-Atom), a novel approach that enables atom-scale and residue-scale unified molecular modeling. ESM-AA achieves this by pretraining on multi-scale code-switch protein sequences and utilizing a multi-scale position encoding to capture relationships among residues and atoms. Experimental results indicate that ESM-AA surpasses previous methods in proteinmolecule tasks, demonstrating the full utilization of protein language models. Further investigations reveal that through unified molecular modeling, ESM-AA not only gains molecular knowledge but also retains its understanding of proteins.1
Matching journals
The top 1 journal accounts for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- CASTER-DTA: Equivariant Graph Neural Networks for Predicting Drug-Target Affinity 96%
- EGRET: Edge Aggregated Graph Attention Networks and Transfer Learning Improve Protein-Protein Interaction Site Prediction 95%
- Leveraging Scaffold Information to Predict Protein-ligand Binding Affinity with an Empirical Graph Neural Network 95%
Similar papers in this journal
- MULAN: Multimodal Protein Language Model for Sequence and Structure Encoding 97%
- SAINT-Angle: self-attention augmented inception-inside-inception network and transfer learning improve protein backbone torsion angle prediction 95%
- FLONE: fully Lorentz network embedding for inferring novel drug targets 95%
Similar papers in this journal
- All-Atom Protein Sequence Design using Discrete Diffusion Models 96%
- Sequence-based Drug-Target Complex Pre-training Enhances Protein-Ligand Binding Process Predictions Tackling Crypticity 95%
- Chemical Genomics Language Model toward Reliable and Explainable Compound-Protein Interaction Exploration 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.