Back

Large Language Models (LLMs) and Empathy - A Systematic Review

Sorin, V.; Brin, D.; Barash, Y.; Konen, E.; Charney, A.; Nadkarni, G.; Klang, E.

2023-08-07 health informatics
10.1101/2023.08.07.23293769 medRxiv
Show abstract

PurposeEmpathy, a cornerstone of human interaction, is a unique quality to humans that Large Language Models (LLMs) are believed to lack. Our study aims to review the literature on the capacity of LLMs in demonstrating empathy MethodsWe conducted a literature search on MEDLINE up to July 2023. Seven publications ultimately met the inclusion criteria. ResultsAll studies included in this review were published in 2023. All studies but one focused on ChatGPT-3.5 by OpenAI. Only one study evaluated empathy based on objective metrics, and all others used subjective human assessment. The studies reported LLMs to exhibits elements of empathy, including emotions recognition and providing emotionally supportive responses in diverse contexts, most of which were related to healthcare. In some cases, LLMs were observed to outperform humans in empathy-related tasks. ConclusionLLMs demonstrated some aspects of empathy in variable scenarios, mainly related to healthcare. The empathy may be considered "cognitive" empathy. Social skills are a fundamental aspect of intelligence, thus further research is imperative to enhance these skills in AI.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

1
PLOS Digital Health
● 106 papers in training set
Top 0.2%
18.6%
Similar papers in this journal
2
PLOS ONE
● 5266 papers in training set
Top 14%
12.5%
Similar papers in this journal
3
Journal of Medical Internet Research
◐ 87 papers in training set
Top 0.1%
10.7%
Similar papers in this journal
4
BMC Medical Education
○ 21 papers in training set
Top 0.2%
7.9%
Similar papers in this journal
5
JMIR Formative Research
◐ 33 papers in training set
Top 0.2%
7.3%
Similar papers in this journal
50% of probability mass above
6
Frontiers in Psychiatry
○ 87 papers in training set
Top 0.6%
3.3%
7
DIGITAL HEALTH
○ 17 papers in training set
Top 0.2%
2.7%
8
International Journal of Medical Informatics
○ 26 papers in training set
Top 0.5%
2.4%
9
Scientific Reports
○ 3612 papers in training set
Top 41%
2.4%
10
Frontiers in Digital Health
○ 24 papers in training set
Top 0.4%
2.4%
11
JMIR Research Protocols
◐ 21 papers in training set
Top 0.6%
1.7%
12
BMJ Health & Care Informatics
● 15 papers in training set
Top 0.7%
1.5%
13
Computers in Biology and Medicine
○ 128 papers in training set
Top 3%
1.5%
14
JMIRx Med
◐ 32 papers in training set
Top 1%
1.4%
15
International Journal of Environmental Research and Public Health
○ 128 papers in training set
Top 4%
1.1%
16
npj Digital Medicine
○ 118 papers in training set
Top 3%
1.0%
17
Frontiers in Public Health
○ 148 papers in training set
Top 5%
1.0%
18
BJPsych Open
● 29 papers in training set
Top 0.7%
0.8%
19
Journal of Clinical and Translational Science
◐ 14 papers in training set
Top 0.5%
0.8%
20
JAMIA Open
◐ 42 papers in training set
Top 2%
0.8%
21
BMC Medical Research Methodology
○ 47 papers in training set
Top 2%
0.6%
22
Healthcare
○ 17 papers in training set
Top 1%
0.6%
23
Philosophical Transactions of the Royal Society B
● 51 papers in training set
Top 1%
0.6%
24
BMJ Open
● 601 papers in training set
Top 13%
0.6%
25
Artificial Intelligence in Medicine
○ 17 papers in training set
Top 0.8%
0.6%
26
Biology Methods and Protocols
◐ 61 papers in training set
Top 3%
0.6%
27
Journal of the American Medical Informatics Association
◐ 71 papers in training set
Top 2%
0.6%
28
Life
○ 29 papers in training set
Top 1%
0.6%
29
BMC Medical Informatics and Decision Making
○ 43 papers in training set
Top 2%
0.6%

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.