Back

DS4DH at MEDIQA-Chat 2023: Leveraging SVM and GPT-3 Prompt Engineering for Medical Dialogue Classification and Summarization

Zhang, B.; Mishra, R.; Teodoro, D.

2023-06-12 health informatics
10.1101/2023.06.08.23291121 medRxiv
Show abstract

This paper presents the results of the Data Science for Digital Health (DS4DH) group in the MEDIQA-Chat Tasks at ACL-ClinicalNLP 2023. Our study combines the power of a classical machine learning method, Support Vector Machine, for classifying medical dialogues, along with the implementation of oneshot prompts using GPT-3.5. We employ dialogues and summaries from the same category as prompts to generate summaries for novel dialogues. Our findings exceed the average benchmark score, offering a robust reference for assessing performance in this field.

Matching journals

The top 6 journals account for 50% of the predicted probability mass.

1
Artificial Intelligence in Medicine
○ 17 papers in training set
Top 0.1%
12.7%
2
Computers in Biology and Medicine
○ 128 papers in training set
Top 0.1%
11.7%
3
PLOS Digital Health
● 106 papers in training set
Top 0.6%
8.7%
4
Journal of Biomedical Informatics
○ 47 papers in training set
Top 0.2%
6.6%
5
Journal of Medical Internet Research
◐ 87 papers in training set
Top 0.3%
6.1%
6
BMC Medical Informatics and Decision Making
○ 43 papers in training set
Top 0.3%
6.1%
50% of probability mass above
7
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 0.6%
5.4%
8
Frontiers in Digital Health
○ 24 papers in training set
Top 0.2%
5.1%
9
Scientific Reports
○ 3612 papers in training set
Top 21%
4.7%
10
IEEE Journal of Biomedical and Health Informatics
● 37 papers in training set
Top 0.4%
3.2%
11
Bioinformatics
◐ 1204 papers in training set
Top 6%
3.0%
12
JAMIA Open
◐ 42 papers in training set
Top 0.7%
2.3%
13
Frontiers in Psychiatry
○ 87 papers in training set
Top 1%
2.1%
14
International Journal of Medical Informatics
○ 26 papers in training set
Top 0.8%
1.7%
15
DIGITAL HEALTH
○ 17 papers in training set
Top 0.5%
1.6%
16
BMJ Health & Care Informatics
● 15 papers in training set
Top 0.8%
1.1%
17
npj Digital Medicine
○ 118 papers in training set
Top 3%
1.1%
18
PLOS ONE
● 5266 papers in training set
Top 56%
1.1%
19
iScience
○ 1154 papers in training set
Top 27%
1.1%
20
JMIR Medical Informatics
◐ 18 papers in training set
Top 0.8%
1.0%
21
JMIR Research Protocols
◐ 21 papers in training set
Top 1%
1.0%
22
Biology Methods and Protocols
◐ 61 papers in training set
Top 2%
0.8%
23
Expert Systems with Applications
○ 11 papers in training set
Top 0.4%
0.8%
24
BMC Bioinformatics
○ 457 papers in training set
Top 6%
0.8%
25
JMIR Formative Research
◐ 33 papers in training set
Top 2%
0.8%
26
Bioinformatics Advances
◐ 203 papers in training set
Top 5%
0.8%
27
Computer Methods and Programs in Biomedicine
○ 28 papers in training set
Top 1%
0.6%