Back

Cross-Sectional Analysis: Interpretation Of Non-Statistically Significant Results In Randomised Controlled Clinical Trials In Rehabilitation

Mugnai, C.; Falsiroli Maistrello, L.; Fiacca, G.; Perucchini, M.; Corbetta, N.; Amateis, F.; Salvioli, S.

2024-09-16 rehabilitation medicine and physical therapy
10.1101/2024.09.16.24313294 medRxiv
Show abstract

IntroductionDespite the CONSORT guidelines, which aim to improve the quality of studies, authors often formulate conclusions based on the dichotomous distinction of the p-value, declaring differences between statistically significant and non-significant. This approach confuses the identification of the real efficacy of the studied treatment. To solve this problem, CONSORT guidelines recommend using confidence intervals, which offer a more complete view of possible effects. However, authors conclusions often remain based on a binary approach, confusing the absence of evidence with the evidence of absence. This error can influence clinical practice and future research, leading to the identification of negative treatments based on statistical insignificance, which reflects a lack of evidence of absence, not the absence of evidence. ObjectivesTo assess the prevalence of misinterpretation of non-statistically significant results, both in the abstract and in the article, in a sample of all randomised controlled trials (RCTs) with non-statistically significant primary outcomes published in 5 rehabilitation journals with the highest impact factor (IF) published between 2019 and 2023 and to assess whether the primary outcome result is reported according to CONSORT guidelines. MethodsWe will conduct a cross-sectional analysis of all Rcts with non-statistically significant primary outcomes in 5 general rehabilitation journals with the highest IF published between 2019 and 2023. We will determine the prevalence of trials in which non-significance is interpreted as absence of evidence, evidence of absence, or advice to use the intervention in clinical practice in the abstract and article conclusions, and the prevalence of trials that adhered to CONSORT guidelines for reporting the primary outcome.

Matching journals

The top 1 journal accounts for 50% of the predicted probability mass.

50% of probability mass above

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.