Authority Signals in AI Cited Health Sources: A Framework for Evaluating Source Credibility in ChatGPT Responses
Jacques, E.; Datuowei, E.; Jones, V.; Basch, C.; Vanderpool, C.; Udeozo, N.; Chapa, G.
Show abstract
Health information seeking has fundamentally changed since the onset of Large Language Models (LLM), with nearly one third of ChatGPTs 800 million users asking health questions weekly. Understanding the sources of those AI generated responses is vital, as health organizations and providers are also investing in digital strategies to organically improve their ranking, reach and visibility in LLM systems like ChatGPT. As AI search optimization strategies are gaining maturity, this study introduces an Authority Signals Framework, organized in four domains that reflect key components to health information seeking, starting with "Who wrote it ?" (Author Credentials), followed by "Who published it?" (Institutional Affiliation), "How was it vetted?" (Quality Assurance), and "How does AI find it?" (Digital Authority). This descriptive cross-sectional study randomly selected 100 questions from HealthSearchQA which contains 3,173 consumer health questions curated by Google Research from publicly available search engine suggestions. Those questions were entered into ChatGPT 5.2 Pro to record and code the cited sources through the lens of the Authority Signals Frameworks four domains. Descriptive statistics were calculated for all cited sources (n=615), and cross tabulations were conducted to examine distinction among organization types. Over 75% of the sources cited in ChatGPTs health generated responses were from established institutional sources, such as Mayo Clinic, Cleveland Clinic, Wikipedia, National Health Service, PubMed with the remaining citations sourced from alternative health information sources that lacked established institutional backing.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
Similar papers in this journal
- Has the pandemic enhanced and sustained digital health-seeking behaviour? A big data interrupted time-series analysis of Google Trends 94%
- Understanding how the design and implementation of Online Consultations influence primary care outcomes: Systematic review of evidence with recommendations for designers, providers, and researchers 93%
- How the COVID-19 pandemic is favoring the adoption of digital technologies in healthcare: a literature review 93%
Similar papers in this journal
- Introducing the EMPIRE Index: A novel, value-based metric framework to measure the impact of medical publications 94%
- COVID-19-related research data availability and quality according to the FAIR principles: A meta-research study 94%
- Clinical code sets and the problem of redundancy in code set repositories 94%
Similar papers in this journal
- Connecting Artificial Intelligence and Primary Care Challenges: Findings from a Multi-Stakeholder Collaborative Consultation 94%
- Measures of socioeconomic advantage are not independent predictors of support for healthcare AI: subgroup analysis of a national Australian survey 93%
- Cracking the Code: A Scoping Review to Unite Disciplines in Tackling Legal Issues in Health Artificial Intelligence 92%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.