Back

EndoGPT: A Proof-of-concept Large Language Model Based Assistant for the Management of Thyroid Nodules

Shah, M.; Kuo, E. J.; Kuo, J. H.; Hsu, S.; McManus, C.; Liou, R.; Lee, J. A.; Sathe, T. S.

2024-05-31 surgery
10.1101/2024.05.29.24308002 medRxiv
Show abstract

Large language models (LLMs) are increasingly being explored for their potential to simulate clinical reasoning. Here, we demonstrate our initial experience using the GPT-4o LLM along with prompt engineering and knowledge retrieval to develop EndoGPT, a clinical decision support tool for the management of thyroid nodules. In a pilot study of 50 cases, EndoGPT demonstrated an 83% concordance rate with expert surgeons assessments and plans. The highest concordance was in diagnosis (93%), followed by the need for an operation (82%) and type of operation (69%). This work suggests that LLM-based assistants may play a useful role in assisting clinicians in the future.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

50% of probability mass above

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.