Evaluation of symptom checker formats to support health literacy and trust in AI: Results from an online randomised-controlled trial

ObjectivesEvaluate the impact of online symptom checker formats on symptom management knowledge, symptom checker trust and acceptability, and behavioural intentions. DesignTwo 5-arm parallel-group online randomised controlled trials. SettingOnline survey. Participants2110 Australian adults recruited through online research panel in June 2025. Almost half identified as man/male (49%) and woman/female (51%), median age 49 years (IQR=28). Participants viewed a hypothetical health scenario (fever and vomiting) followed by a screenshot of an online symptom checker from national health service provider, healthdirect. InterventionsParticipants were randomised to symptom acuity level (low: self-care at home; or moderate: see a General Practitioner (GP) in 24 hours) and one of five symptom checker formats. The standard format showed the existing healthdirect symptom checker advice. The remaining formats were AI-enhanced versions. These included an AI-enhanced version with e.g. more tailored advice, rationale for acuity level, and AI disclosure statement. The other AI-enhanced formats had additional features: numbered steps, multimedia, and more detailed information about the use of AI. Main outcome measuresPrimary outcomes were intentions to follow the symptom checkers self-care advice and intentions to see a GP in 24 hours. Secondary outcomes were trust in advice, knowledge of symptom management, and acceptability of the tool. All outcomes were assessed immediately post-intervention; knowledge was also assessed after 2 weeks. ResultsWhen advised to self-care at home, the AI-enhanced groups reported lower intentions to see a GP in 24 hours (median 3.00 out of 5), compared to the standard (original) tool (median 4.00; adjusted p = 0.003). There were no other significant effects on intentions. Immediately following the intervention, participants who viewed an AI-enhanced format reported greater knowledge about how to manage current and changing symptoms, across both acuity levels (adjusted ps <0.001). Knowledge gains were not sustained at 2 week follow-up. There were no significant effects on trust or acceptability. ConclusionsParticipants who viewed the more tailored information in the AI-enhanced formats demonstrated stronger knowledge for managing symptoms than those who viewed the standard format. There was also some evidence that an AI-enhanced format may be more effective at reducing use of primary care for symptoms that can be managed at home. Trust and acceptability were high across formats, and the explicit use of AI did not impact significantly on these outcomes. Future research should investigate these formats using interactive prototypes across a wider variety of health contexts. RegistrationACTRN ACTRN12625000474459p Key messagesO_LIWhat is already known on this topic: Although online, evidence-based symptom checkers have been widely available from reputable health organisations for over a decade, they often face poor uptake and may not adequately meet health literacy needs of diverse users. C_LIO_LIWhat this study adds: Symptom checker features that could be implemented with AI, such as tailored information and a clear rationale for triage advice, may help support appropriate symptom management. Statements about the tools use of AI did not appear to impact trust or acceptability of the symptom checker tool. C_LIO_LIHow this study might affect research, practice or policy: Findings from this study suggest that using AI to enhance symptom checker advice may not impact negatively on trust and acceptability of the tool, and may improve appropriate symptom management. Further research is needed to investigate AI-enhanced symptom checker formats using interactive prototypes across a wider variety of health contexts. C_LI