Evolution of Reporting P-values Across the Biomedical Literature, 1990-2025: an Updated Meta-Research Study
Choi, J.; Lee, K.; Chavalarias, D.; Shin, J. I.; Ioannidis, J.
Show abstract
ImportanceOver several decades there have been extensive debates on the use and misuse of statistical significance. It would be important to capture what P-values are reported in biomedical papers and whether their patterns have changed over time. ObjectiveTo quantify the reporting dynamics on P-values in biomedical articles in PubMed and PubMed Central(PMC) database over a 35-year period (1990-2025). DesignData were retrieved from the National Library of Medicine via PubMed and PubMed Central (PMC, full-text articles included), fetching the entire accessible corpus. Records were computationally processed using a regular expression algorithm, validated for various mathematical formats, to extract reported P-values from the text. SettingThe study includes 22,734,796 PubMed abstracts, 6,031,459 PMC abstracts and 6,397,787 PMC full-texts. Main Outcomes and MeasuresProportion of article reporting at least 1 P-value, at least 1 P-value less than thresholds (.05 and .005), distribution of P-values by magnitude and operator type. ResultsThe proportion of articles reporting P-values increased from 7.5% in 1990 to 18.3% in 2025 for PubMed abstracts, and from 5.2% to 53.3% for PMC full-texts. The median number of P-values per article increased from 2 to 7 in PMC full-text articles, with upward trends observed in all databases. A high proportion of P-values remains clustered around .05 and .001 in all databases. The proportion of articles reporting at least one P-value [≤] .05 has remained in the range 94%-98% since 1998, while the proportion reporting at least one P-value[≤] .005 has increased over time, reaching 57.0% for PubMed abstracts and 62.5% for PMC full-texts. The reporting of exact P-values increased until 2015, but with no further increase in the last 10 years (PubMed abstracts: 17.6% in 1990, 51.1% in 2015, 49.8% in 2025) Conclusions and RelevanceOur evaluation demonstrates the pervasive entrenchment of P-values, despite heavy debates and major changes in the content of the biomedical literature over time. More P-values are reported and papers using P-values almost always report some that are statistically significant. Readers should remain aware of the major issues surrounding P-value misuse and misinterpretation. Key pointsO_ST_ABSQuestionC_ST_ABSWith continuing debate regarding the use and misuse of statistical significance, how have reported P-values evolved over the past 35 years? FindingsAcross over 22 million PubMed abstracts and over 6 million full-texts, reporting of P-values became more common over time. Almost all (94-98%) abstracts and full-text reporting P-values have at least one significant at the .05 threshold. The reporting of exact P-values increased until 2015 but plateaued since then. Clustering around traditional statistical significance thresholds remains consistent. MeaningP-values reporting has become more common over time, with pervasive prevalence of significant P-values across the biomedical literature.
Matching journals
The top 8 journals account for 50% of the predicted probability mass.