Science or Advocacy? The Global Rise of Policy Claims in Population Health Research (1990-2024)
Bann, D.; Wang, M.; Davies, N. M.; Wright, L.; O'Connor, M.; Courtin, E.
Show abstract
Should original research routinely contain prominent policy claims, such as recommendations for policymakers or broad calls to action? Growing emphasis on "research impact" might be welcome but also have unintended consequences that include risks of overextrapolation, the blurring of roles between scientists and advocates, and potential erosion of scientific credibility. To inform this debate, we examined 45,807 abstracts from ten leading Epidemiology and Public Health journals (1990-2024). Using a large language model with human validation, we classified policy claims and mapped their prevalence by time, country, journal, field of study, and study design. Policy claims markedly increased in frequency from 17.6% in 1990-1999 to 35.8% in 2020-2024, with wide variation across countries (>40% in Italy and Australia vs <19% in Norway and Japan, 2020-2024) and journals (>60% in some vs <6% in others). Keywords linked to higher claim rates differed by topic and time: some corresponded to topics with clear causal evidence ("tobacco"), others to topics with more complex causal evidence and notable researcher advocacy ("health inequalities" and "COVID-19"). Claims were most common in qualitative or cross-sectional studies, and less common in cohort, quasi-experimental, or experimental studies. We argue that these patterns reflect a research culture increasingly oriented toward claiming policy relevance--and incentives that encourage attaching claims to single studies. Our findings raise questions about how scientists and journals balance evidence, advocacy, and credibility. Ensuring that policy claims remain commensurate with evidence will be central to build trust as policy impact continues to be incentivised. We discuss alternative ways for researchers and publishers to engage meaningfully with policy beyond attaching claims to individual studies, and share our data and scripts to catalyse further work in this area.
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Updating the PRISMA reporting guideline for network meta-analysis: a scoping review 94%
- High-cited favorable studies for COVID-19 treatments ineffective in large trials 93%
- Quantitative bias analysis methods for summary level epidemiologic data in the peer-reviewed literature: a systematic review 93%
Similar papers in this journal
- Evaluation of the sensitivity, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production 95%
- Development of a search filter to retrieve reports of interrupted time series studies from MEDLINE and PubMed 95%
- Investigation of bias due to selective inclusion of study effect estimates in meta-analyses of nutrition research 94%
Similar papers in this journal
- The methodologies to assess the effects of non-pharmaceutical interventions during COVID-19: a systematic review 93%
- Mendelian randomisation for mediation analysis: current methods and challenges for implementation 88%
- Characterising patterns of COVID-19 and long COVID symptoms: Evidence from nine UK longitudinal studies 87%
Similar papers in this journal
- Publishing at any cost: a cross-sectional study of the amount that medical researchers spend on open-access publishing each year 94%
- The incubation period of COVID-19: A rapid systematic review and meta-analysis of observational research 94%
- Comparison of preprints and final journal publications from COVID-19 Studies: Discrepancies in results reporting and spin in interpretation 94%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.