Back

ProGen: Language Modeling for Protein Generation

Madani, A.; McCann, B.; Naik, N.; Keskar, N. S.; Anand, N.; Eguchi, R. R.; Huang, P.; Socher, R.

2020-03-08 synthetic biology
10.1101/2020.03.07.982272 bioRxiv
Show abstract

Generative modeling for protein engineering is key to solving fundamental problems in synthetic biology, medicine, and material science. We pose protein engineering as an unsupervised sequence generation problem in order to leverage the exponentially growing set of proteins that lack costly, structural annotations. We train a 1.2B-parameter language model, ProGen, on [~]280M protein sequences conditioned on taxonomic and keyword tags such as molecular function and cellular component. This provides ProGen with an unprecedented range of evolutionary sequence diversity and allows it to generate with fine-grained control as demonstrated by metrics based on primary sequence similarity, secondary structure accuracy, and conformational energy.

Matching journals

The top 4 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 0.4%
34.6%
Similar papers in this journal
2
PLOS Computational Biology
● 1863 papers in training set
Top 5%
6.8%
Similar papers in this journal
3
Nature Machine Intelligence
○ 70 papers in training set
Top 0.5%
5.2%
Similar papers in this journal
4
Nature Communications
○ 5641 papers in training set
Top 32%
4.1%
Similar papers in this journal
50% of probability mass above
5
Protein Engineering, Design and Selection
◐ 15 papers in training set
Top 0.1%
3.5%
Similar papers in this journal
6
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 1%
3.3%
7
Protein Science
○ 246 papers in training set
Top 1%
3.3%
8
Briefings in Bioinformatics
◐ 354 papers in training set
Top 3%
3.3%
9
Cell Systems
○ 201 papers in training set
Top 2%
2.1%
10
Scientific Reports
○ 3612 papers in training set
Top 45%
2.1%
11
Nature Computational Science
○ 55 papers in training set
Top 0.4%
2.1%
12
Journal of Molecular Biology
○ 232 papers in training set
Top 2%
1.7%
13
ACS Synthetic Biology
● 287 papers in training set
Top 1%
1.7%
14
Nucleic Acids Research
◐ 1281 papers in training set
Top 8%
1.7%
15
Journal of Chemical Information and Modeling
● 238 papers in training set
Top 2%
1.7%
16
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 27%
1.7%
17
Bioinformatics Advances
◐ 203 papers in training set
Top 3%
1.7%
18
PeerJ
◐ 308 papers in training set
Top 7%
1.3%
19
ACS Omega
● 105 papers in training set
Top 2%
1.1%
20
BMC Bioinformatics
○ 457 papers in training set
Top 5%
1.1%
21
eLife
● 5828 papers in training set
Top 62%
1.0%
22
iScience
○ 1154 papers in training set
Top 35%
0.9%
23
Journal of Cheminformatics
○ 29 papers in training set
Top 0.7%
0.9%
24
Synthetic Biology
◐ 24 papers in training set
Top 0.3%
0.9%
25
Nature Methods
○ 385 papers in training set
Top 6%
0.8%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.