Information Leaflets vs Artificial Intelligence: Comparing Perceptions of Stroke Survivors and Professionals in a Mixed-methods Study

BackgroundStroke survivors often describe problems of insufficient access to information post-discharge. Traditional resources may not meet their information needs, but Artificial Intelligence (AI) could play a role. AimsTo compare user perceptions of stroke information from third sector stroke websites with that generated by AI and summarize the attributes of the preferred stroke information formats. MethodsUK third sector stroke websites were searched for materials relevant to 15 questions commonly asked by stroke survivors. ChatGPT(-4o) was used to generate responses to these questions. Stroke professionals (clinicians, researchers), stroke survivors, and their caregivers reviewed third sector and AI responses, indicating the source of the response and their preferred text. Participants also rated responses on scales of empathy, trustworthiness, reliability, comprehensibility and usefulness, and provided free text comments. Proportions of preference and correctly guessed responses, as well as mean ratings, were compared between the groups. Framework analysis was used to identify the attributes of response formats preferred by stroke survivors. ResultsRelevant responses were found for 13 (87%) out of 15 questions. Across groups, 60 participants with mean age of 44 (SD=14) and 57% females, correctly identified 184/300(61%) of AI responses, and preferred AI response in 123/300(46%) of the cases. The groups differed in their preference with clinicians being least likely to choose AI (34%), followed by stroke survivors (49%) and researchers (54%). All groups viewed third sector responses as more empathetic in tone. The themes of content, structure, and tone of responses were described by stroke survivors with the emphasis on clarity, conciseness, and approachable tone. ConclusionsAI-generated responses to stroke questions were rated positively by stroke survivors and researchers, whereas stroke clinicians were more sceptical. Given that stroke information materials are intended for people with lived experience of stroke, their input should be prioritized to inform development of new resources.