ESMBind and QBind: LoRA, QLoRA, and ESM-2 for Predicting Binding Sites and Post Translational Modification
Schreiber, A.
Show abstract
In this study we discuss the viability of applying protein language models to the problem of predicting bindings sites of protein sequences from single sequences alone using Low Rank Adaptation (LoRA) and Quantized Low Rank Adaptation (QLoRA). No Multiple Sequence Alignment (MSA) or structural information for the proteins was used. Moreover, using LoRA and QLoRA shows improved performance over vanilla full finetuning, and significantly helps in mitigating overfitting. Also, due to the efficiency of LoRA and QLoRA, we are able to train the larger ESM-2 models on modest hardware, making the method very attractive and accessible. We also note that this technique serves as an important regularization technique and serves to improve generalization of models on unseen data.
Matching journals
The top 9 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
- Classification of protein binding ligands using structural dispersion of binding site atoms from principal axes 95%
- Leveraging large language models for data analysis automation 93%
- Countering reproducibility issues in mathematical models with software engineering techniques: A case study using a one-dimensional mathematical model of the atrioventricular node 92%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.