Back

Cell2Sentence: Teaching Large Language Models the Language of Biology

Levine, D.; Rizvi, S. A.; Levy, S.; Pallikkavaliyaveetil MohammedSheriff, N.; Wu, R.; Han, I.; Zhang, Z.; Fonseca, A.; Chen, X.; Ghadermarzi, S.; Karbasi, A.; Dhodapkar, R. M.; van Dijk, D.

2023-11-29 bioinformatics
10.1101/2023.09.11.557287 bioRxiv
Show abstract

We introduce Cell2Sentence (C2S), a novel method to directly adapt large language models to a biological context, specifically single-cell transcriptomics. By transforming gene expression data into "cell sentences," C2S bridges the gap between natural language processing and biology. We demonstrate cell sentences enable the fine-tuning of language models for diverse tasks in biology, including cell generation, complex cell-type annotation, and direct data-driven text generation. Our experiments reveal that GPT-2, when fine-tuned with C2S, can generate biologically valid cells based on cell type inputs, and accurately predict cell types from cell sentences. This illustrates that language models, through C2S fine-tuning, can acquire a significant understanding of single-cell biology while maintaining robust text generation capabilities. C2S offers a flexible, accessible framework to integrate natural language processing with transcriptomics, utilizing existing models and libraries for a wide range of biological applications.

Matching journals

The top 4 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 0.6%
30.8%
Similar papers in this journal
2
Briefings in Bioinformatics
◐ 354 papers in training set
Top 1%
6.7%
Similar papers in this journal
3
Bioinformatics Advances
◐ 203 papers in training set
Top 0.7%
6.7%
Similar papers in this journal
4
Patterns
○ 78 papers in training set
Top 0.2%
6.2%
Similar papers in this journal
50% of probability mass above
5
Cell Systems
○ 201 papers in training set
Top 0.7%
5.4%
Similar papers in this journal
6
Nature Communications
○ 5641 papers in training set
Top 30%
4.4%
7
Nature Machine Intelligence
○ 70 papers in training set
Top 0.6%
4.3%
8
Genome Biology
○ 637 papers in training set
Top 3%
4.0%
9
PLOS Computational Biology
● 1863 papers in training set
Top 11%
2.8%
10
iScience
○ 1154 papers in training set
Top 14%
2.1%
11
Cell Reports Methods
○ 165 papers in training set
Top 2%
1.7%
12
BMC Bioinformatics
○ 457 papers in training set
Top 4%
1.3%
13
Nature Methods
○ 385 papers in training set
Top 5%
1.3%
14
Genome Research
● 468 papers in training set
Top 5%
1.1%
15
Frontiers in Genetics
○ 230 papers in training set
Top 4%
1.1%
16
Nucleic Acids Research
◐ 1281 papers in training set
Top 11%
1.1%
17
Proceedings of the National Academy of Sciences
● 2444 papers in training set
Top 39%
1.0%
18
eLife
● 5828 papers in training set
Top 63%
1.0%
19
Molecular Biology of the Cell
● 311 papers in training set
Top 3%
0.9%
20
Journal of Computational Biology
○ 48 papers in training set
Top 1%
0.8%
21
GigaScience
◐ 212 papers in training set
Top 5%
0.6%
22
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 8%
0.6%
23
NAR Genomics and Bioinformatics
◐ 242 papers in training set
Top 5%
0.6%
24
Scientific Reports
○ 3612 papers in training set
Top 77%
0.6%
25
BioData Mining
○ 22 papers in training set
Top 0.9%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.