MAPEVAScR: a four-phase framework integrating transparent rule based text mining, evidence gap mapping and a mandatory human validation gate for scoping reviews framework development and empirical validation in occupational health.
ERAS LEVANO, C. J.
Show abstract
Background. Automation is increasingly used to reduce the screening workload of evidence syntheses, but its error is real, variable and rarely reported. The 2025 joint position statement of Cochrane, Campbell, JBI and the Collaboration for Environmental Evidence mandates human oversight of artificial intelligence in evidence synthesis, yet, as of today, no international guideline requires quantitative measurement and reporting of automated-screening error against a human gold standard. Methods. We developed MAPEVAScR (MAPeo de EVidencia con VAlidacion humana), a four phase framework for scoping reviews: (A) datalake construction with deterministic, traceable deduplication; (B) transparent, record level auditable rule based text mining with confidence tiers; (C) evidence gap mapping that feeds back human screening prioritisation; and (D) a mandatory human validation gate verifying 100% of the candidate corpus and issuing a quantitative screening error report (per tier precision, confusion matrix, falsepositive families, documented changes to conclusions). The framework was empirically validated on a scoping review of epilepsy and occupational fitness (2015 to 2025; PubMed, Scopus, Web of Science): a funnel of 2,008 [->] 1,435 [->] 1,114 [->] 230 records was reduced to 127 human confirmed inclusions (present study). Results. The rule based filter achieved a global precision of 55.2% (Tier A 75.5%, Tier B 37.5%, Tier C 45.8%); human validation identified 103 false positives classifiable into seven reproducible error families, and 107 of 127 included records required human correction of automated thematic labels. Without the gate, the review's central thematic conclusion would have been published reversed: automated labelling ranked Aptitude/employability third (12.6% of records), whereas after validation it ranked first (73.2%) (present study). Conclusions. A mandatory human validation gate with quantitative error metrics closes a reporting gap that current guidelines leave open: in the present study it prevented publication of an inverted synthesis. We propose the screening error report as a minimum reporting standard for any evidence synthesis using automated screening. Keywords: scoping review; text mining; human validation; automation bias; evidence gap map; occupational health; epilepsy; PRISMAScR
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Is artificial intelligence for medical professionals serving the patients? Protocol for a mixed method systematic review on patient-relevant benefits and harms of algorithmic decision-making 92%
- Evidence-Based, Cost-Effective Interventions To Suppress The COVID-19 Pandemic: A Systematic Review 90%
- To Include or Not to Include? A prescription from the pharmacy on how to use active learning assisted screening in systematic reviews 90%
Similar papers in this journal
- The Use of Machine Learning in Occupational Risk Communication for Healthcare Workers – Protocol for scoping review 93%
- Strategies used to manage overlap of primary study data by exercise-related overviews. Protocol for a systematic methodological review 91%
- GPT for RCTs?: Using AI to measure adherence to reporting guidelines 91%
Similar papers in this journal
- Modelling the impact of behavioural interventions during pandemics: A systematic review 92%
- Wellbeing Impact Study of High-Speed 2 (WISH2): Protocol for a mixed-methods examination of the impact of major transport infrastructure development on mental health and wellbeing 91%
- Menopause education of healthcare professionals: A scoping review protocol 91%
Similar papers in this journal
- HIPTox – Hazard Identification Platform to Assess the Health Impacts from Indoor and Outdoor Air Pollutant Exposures, through Mechanistic Toxicology: A single-centre double-blind human exposure trial protocol 90%
- Quality assessment of studies included in Cochrane oral health systematic reviews 90%
- The Italian National Surveillance System for Occupational Injuries: Conceptual Framework and Fatal Outcomes, 2002-2016. 89%
Similar papers in this journal
- Covid-19 Risk by work-related factors: Pooled analysis of individual linked data from 14 cohorts 90%
- Differences in COVID-19 vaccination coverage by occupation in England: a national linked data study 89%
- Occupation, Work-Related Contact, and SARS-CoV-2 Anti-Nucleocapsid Serological Status: Findings from the Virus Watch prospective cohort study 88%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.