Mut-BPE: A Modified BPE Strategy Improves Variant Effect Prediction
Xu, Y.; Long, W.; Lu, Y.; Xu, Y.; Zhang, Y.
Show abstract
Byte Pair Encoding (BPE) is widely used in genome foundation models for its ability to compress long DNA sequences into fewer tokens. However, its variable-length tokens often span multiple nucleotides, limiting the models sensitivity to single-nucleotide variations--an essential requirement for accurate Variant Effect Prediction (VEP). We introduce Mut-BPE, a training-free, plug-and-play tokenization strategy that augments BPE with explicit single-nucleotide resolution at variant sites. Mut-BPE preserves the efficiency of BPE while enhancing its ability to represent subtle genomic alterations. To evaluate its effectiveness, we applied Mut-BPE to DNABERT-2 and conducted extensive experiments across six diverse datasets spanning gene expression, pathogenicity, and trait-associated variants, under zero-shot and fine-tuning settings. Mut-BPE consistently outperformed conventional BPE tokenization, yielding significant improvements in both AUROC and AUPRC, particularly in imbalanced datasets. These results highlight Mut-BPE as a practical enhancement for genomic foundation models in VEP tasks. Code availability: https://anonymous.4open.science/r/Mut-BPE-1182
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- seqgra: Principled Selection of Neural Network Architectures for Genomics Prediction Tasks 96%
- Graph Convolutional Networks for Epigenetic State Prediction Using Both Sequence and 3D Genome Data 96%
- Exploiting deep transfer learning for the prediction of functional noncoding variants using genomic sequence 95%
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.