Back

A generative foundation model for antibody sequence understanding

Barton, J.; Gaspariunas, A.; Yadin, D. A.; Dias, J.; Nice, F. L.; Minns, D. H.; Snudden, O.; Povall, C.; Valle Tomas, S.; Dobson, H.; Farmery, J. H.; Leem, J.; Galson, J. D.

2024-07-24 bioinformatics
10.1101/2024.05.22.594943 bioRxiv
Show abstract

Here we introduce FAbCon, a generative antibody-specific language model comprising 2.4 billion parameters. A commonly accepted wisdom in developing large language models is that increasing model scale will translate to higher performance on downstream tasks. Starting from a 144-million parameter setup, we show that progressively larger models achieve greater accuracy in predicting antigen binding and can also be used to design new antibodies with good predicted developability potential. FAbCon is available on huggingface.co/alchemab.

Matching journals

The top 3 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 0.5%
31.1%
Similar papers in this journal
2
mAbs
○ 32 papers in training set
Top 0.1%
15.1%
Similar papers in this journal
3
Nature Machine Intelligence
○ 70 papers in training set
Top 0.1%
11.0%
Similar papers in this journal
50% of probability mass above
4
Bioinformatics Advances
◐ 203 papers in training set
Top 0.4%
7.9%
Similar papers in this journal
5
Cell Systems
○ 201 papers in training set
Top 0.7%
5.5%
Similar papers in this journal
6
Patterns
○ 78 papers in training set
Top 0.3%
4.3%
7
Nature Methods
○ 385 papers in training set
Top 2%
4.0%
8
Briefings in Bioinformatics
◐ 354 papers in training set
Top 3%
2.6%
9
PLOS Computational Biology
● 1863 papers in training set
Top 13%
1.9%
10
Nature Communications
○ 5641 papers in training set
Top 45%
1.7%
11
Nature Biotechnology
○ 172 papers in training set
Top 3%
1.3%
12
Cell Reports Methods
○ 165 papers in training set
Top 2%
1.1%
13
ImmunoInformatics
○ 12 papers in training set
Top 0.2%
1.0%
14
eLife
● 5828 papers in training set
Top 66%
0.8%
15
Cell Genomics
○ 172 papers in training set
Top 4%
0.8%
16
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 7%
0.8%
17
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 41%
0.8%
18
iScience
○ 1154 papers in training set
Top 41%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.