Back

ChatGPT Decision Support System: Utility in Creating Public Policy for Concussion/Repetitive Brain Trauma Associated With Neurodegenerative Diseases

Comrie, D.

2023-04-18 neurology
10.1101/2023.04.17.23288681 medRxiv
Show abstract

This article evaluates the ChatGPT decision support systems utility for creating policies related to concussion and repetitive brain trauma associated with neurodegenerative disease risk. It is generally stable and fast. prompt/response pairs (n=259) were examined returning: six prompt response pairs that regenerated (2.31%); one Incorrect Answer; (.38%) one fragment (.38%). Its accuracy, validity, opacity, informational latency and vulnerability to manipulation limits its utility. ChatGPTs data can be both out-of-date and incomplete which limits its utility use to subject matter experts analyzing expert statements. ChatGPTs performance is affected by prompts involving stakeholder bias and litigation management, such as race. Nonetheless, ChatGPT demonstrated its ability to respond in both American and British/Australian English with ease. Overall, this study suggests that ChatGPT has limitations that need to be addressed before it can be widely used in decision-making related to concussion and repetitive brain trauma policies.

Matching journals

The top 5 journals account for 50% of the predicted probability mass.

50% of probability mass above

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.