Text Mining Drug-Protein Interactions using an Ensemble of BERT, Sentence BERT and T5 models
Sui, X.; Wang, W.; Zhang, J.
Show abstract
In this work, we trained an ensemble model for predicting drug-protein interactions within a sentence based on only its semantics. Our ensembled model was built using three separate models: 1) a classification model using a fine-tuned BERT model; 2) a fine-tuned sentence BERT model that embeds every sentence into a vector; and 3) another classification model using a fine-tuned T5 model. In all models, we further improved performance using data augmentation. For model 2, we predicted the label of a sentence using k-nearest neighbors with its embedded vector. We also explored ways to ensemble these 3 models: a) we used the majority vote method to ensemble these 3 models; and b) based on the HDBSCAN clustering algorithm, we trained another ensemble model using features from all the models to make decisions. Our best model achieved an F-1 score of 0.753 on the BioCreative VII Track 1 test dataset.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- A Sequence Labeling Framework for Extracting Drug-Protein Relations from Biomedical Literature 98%
- LSD600: the first corpus of biomedical abstracts annotated with lifestyle–disease relations 96%
- RegulaTome: a corpus of typed, directed, and signed relations between biomedical entities in the scientific literature 95%
Similar papers in this journal
- Mining drug-target interactions from biomedical literature using chemical and gene descriptions-based ensemble transformer model. 95%
- Improving protein function prediction by learning and integrating representations of protein sequences and function labels 95%
- CoNECo: A Corpus for Named Entity recognition and normalization of protein Complexes 94%
Similar papers in this journal
- SKiM-GPT: Combining Biomedical Literature-Based Discovery with Large Language Model Hypothesis Evaluation 96%
- COMIC: Explainable Drug Repurposing via Contrastive Masking for Interpretable Connections 95%
- Relation extraction between bacteria and biotopes from biomedical texts with attention mechanisms and domain-specific contextual representations 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.