Changing patterns in reporting and sharing of review data in systematic reviews with meta-analysis of the effects of interventions: a meta-research study from the REPRISE project
Nguyen, P.-Y.; Kanukula, R.; McKenzie, J. E.; Alqaidoom, Z.; Brennan, S. E.; Haddaway, N. R.; Hamilton, D. G.; Karunananthan, S.; McDonald, S.; Moher, D.; Nakagawa, S.; Nunan, D.; Tugwell, P.; Welch, V. A.; Page, M. J.
Show abstract
ObjectivesTo examine changes in completeness of reporting and frequency of sharing data, analytic code and other review materials in systematic reviews (SRs) over time; and factors associated with these changes. DesignCross-sectional meta-research study. SampleA random sample of 300 SRs with meta-analysis of aggregate data on the effects of a health, social, behavioural or educational intervention, which were indexed in PubMed, Science Citation Index, Social Sciences Citation Index, Scopus and Education Collection in November 2020. Analysis/OutcomesThe extent of complete reporting and frequency of sharing review materials in these reviews were compared with 110 SRs indexed in February 2014. Associations between completeness of reporting and various factors (e.g. self-reported use of reporting guidelines, journals data sharing policies) were examined by calculating risk ratios (RR) and 95% confidence intervals (CI). ResultsSeveral items were reported sub-optimally among 300 SRs from 2020, such as a registration record for the review (38%), a full search strategy for at least one database (71%), methods used to assess risk of bias (62%), methods used to prepare data for meta-analysis (34%), and funding source for the review (72%). Only a few items not already reported at a high frequency in 2014 were reported more frequently in 2020. There was no evidence that reviews using a reporting guideline were more completely reported than reviews not using a guideline. Reviews published in 2020 in journals that mandated either data sharing or inclusion of Data Availability Statements were more likely to share their review materials (e.g. data, code files) (18% vs 2%). ConclusionIncomplete reporting of several recommended items for systematic reviews persists, even in reviews that claim to have followed a reporting guideline. Data sharing policies of journals potentially encourage sharing of review materials. O_TEXTBOXO_TEXTBOXNOSUMMARY BOXC_TEXTBOXNO What is already known on this topicO_LIComplete reporting of methods and results, as well as sharing data and analytic code, enhances transparency and reproducibility of systematic reviews. The extent of complete reporting and sharing of data or analytic code among systematic reviews needs to be comprehensively assessed. C_LIO_LIUse of reporting guidelines, which are designed to improve reporting in systematic reviews, is increasing. It is unclear whether this increase has had an impact on reporting of methods and results in systematic reviews. C_LIO_LIMore journals are adopting open data policies which aim to promote data sharing. The impact of these policies on sharing data and analytic code in systematic reviews is also unclear. C_LI What this study addsO_LIIncomplete reporting of several recommended items in systematic reviews persists. Frequency of sharing review data and analytic code is currently low (7%). C_LIO_LIAn increase in self-reported use of a reporting guideline was observed between 2014-2020; however, there was no evidence that reviews using a reporting guideline were more completely reported than reviews not using a guideline. C_LIO_LIReviews published in 2020 in journals that mandated either data sharing or inclusion of Data Availability Statements were more likely to share their review materials (e.g. data, code files). C_LI C_TEXTBOX
Matching journals
The top 3 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Evaluation of the sensitivity, accuracy and currency of the Cochrane COVID-19 Study Register for supporting rapid evidence synthesis production 97%
- Synthesizing evidence from the earliest studies to support decision-making: to what extent could the evidence be reliable? 97%
- Development of a search filter to retrieve reports of interrupted time series studies from MEDLINE and PubMed 96%
Similar papers in this journal
- Characteristics and completeness of reporting of systematic reviews of prevalence studies in adult populations: a meta-epidemiological study 98%
- Updating the PRISMA reporting guideline for network meta-analysis: a scoping review 97%
- Methods used to select results to include in meta-analyses of nutrition research: a meta-research study 97%
Similar papers in this journal
- Modelling the impact of behavioural interventions during pandemics: A systematic review 95%
- COVID-19 and pregnancy: An umbrella review of clinical presentation, vertical transmission, and maternal and perinatal outcomes 94%
- COVID-19-related research data availability and quality according to the FAIR principles: A meta-research study 93%
Similar papers in this journal
- Evidence-Based, Cost-Effective Interventions To Suppress The COVID-19 Pandemic: A Systematic Review 92%
- To Include or Not to Include? A prescription from the pharmacy on how to use active learning assisted screening in systematic reviews 92%
- Repurposing Existing Medications for Coronavirus Disease 2019: Protocol for a Rapid and Living Systematic Review 91%
Similar papers in this journal
- Protocol for the development of a tool (INSPECT-SR) to identify problematic randomised controlled trials in systematic reviews of health interventions 96%
- Strategies used to manage overlap of primary study data by exercise-related overviews. Protocol for a systematic methodological review 96%
- Comparison of preprints and final journal publications from COVID-19 Studies: Discrepancies in results reporting and spin in interpretation 95%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.