Back

Enhancing Biomarker-Based Oncology Trial Matching Using Large Language Models

Al Khoury, N.; Shaik, M.; Wurmus, R.; Akalin, A.

2024-09-19 bioinformatics
10.1101/2024.09.13.612922 bioRxiv
Show abstract

Clinical trials are an essential component of drug development for new cancer treatments, yet the information required to determine a patients eligibility for enrollment is scattered in large amounts of unstructured text. Genomic biomarkers are especially important in precision medicine and targeted therapies, making them essential for matching patients to appropriate trials. Large language models (LLMs) offer a promising solution for extracting this information from clinical trial data, aiding both physicians and patients in identifying suitable matches. In this study, we explore various LLM strategies for extracting genetic biomarkers from oncology trials to improve patient enrollment rates. Our results show that open-source language models, when applied out-of-the-box, effectively capture complex logical expressions and structure genomic biomarkers in disjunctive normal form, outperforming closed-source models such as GPT-4 and GPT-3.5-Turbo. Furthermore, fine-tuning these open-source models with additional data significantly enhances their performance.

Matching journals

The top 10 journals account for 50% of the predicted probability mass.

1
Bioinformatics
◐ 1204 papers in training set
Top 3%
7.1%
Similar papers in this journal
2
JCO Clinical Cancer Informatics
● 22 papers in training set
Top 0.1%
6.6%
Similar papers in this journal
3
Scientific Reports
○ 3612 papers in training set
Top 11%
6.6%
Similar papers in this journal
4
npj Digital Medicine
○ 118 papers in training set
Top 1%
5.4%
Similar papers in this journal
5
Nature Communications
○ 5641 papers in training set
Top 27%
5.4%
Similar papers in this journal
6
Scientific Data
○ 209 papers in training set
Top 0.5%
5.0%
7
iScience
○ 1154 papers in training set
Top 4%
4.2%
8
Patterns
○ 78 papers in training set
Top 0.4%
3.9%
9
Briefings in Bioinformatics
◐ 354 papers in training set
Top 3%
3.1%
10
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 1%
3.1%
50% of probability mass above
11
GigaScience
◐ 212 papers in training set
Top 1%
3.0%
12
Nature Machine Intelligence
○ 70 papers in training set
Top 1.0%
2.7%
13
PLOS Computational Biology
● 1863 papers in training set
Top 12%
2.3%
14
Computational and Structural Biotechnology Journal
● 242 papers in training set
Top 3%
2.1%
15
Journal of Biomedical Informatics
○ 47 papers in training set
Top 0.8%
1.9%
16
BMC Bioinformatics
○ 457 papers in training set
Top 4%
1.9%
17
Journal of Medical Internet Research
◐ 87 papers in training set
Top 2%
1.7%
18
BioData Mining
○ 22 papers in training set
Top 0.2%
1.7%
19
Computers in Biology and Medicine
○ 128 papers in training set
Top 3%
1.7%
20
Journal of Chemical Information and Modeling
● 238 papers in training set
Top 2%
1.7%
21
JAMIA Open
◐ 42 papers in training set
Top 1.0%
1.6%
22
IEEE Transactions on Computational Biology and Bioinformatics
● 20 papers in training set
Top 0.3%
1.5%
23
Bioinformatics Advances
◐ 203 papers in training set
Top 3%
1.5%
24
Nucleic Acids Research
◐ 1281 papers in training set
Top 10%
1.5%
25
Artificial Intelligence in Medicine
○ 17 papers in training set
Top 0.4%
1.4%
26
IEEE/ACM Transactions on Computational Biology and Bioinformatics
● 38 papers in training set
Top 0.7%
1.4%
27
PLOS ONE
● 5266 papers in training set
Top 54%
1.1%
28
Genome Biology
○ 637 papers in training set
Top 7%
1.1%
29
BMC Medical Informatics and Decision Making
○ 43 papers in training set
Top 2%
1.0%
30
Artificial Intelligence in the Life Sciences
○ 13 papers in training set
Top 0.2%
1.0%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.