Does a new randomized trial change the evidence? A pilot audit of evidence deltas in colorectal surgery

Background. Research output is counted in publications, although publications differ in how much uncertainty they resolve. An evidence delta is the change between the evidence state immediately before an index trial's results are published and the evidence state after. Whether such a comparison can be reconstructed from the published record is unknown. Methods. Six colorectal-surgery evidence lineages were chosen by the first author before any outcome data were seen, each defined by one binary patient-relevant outcome. Randomized trials of any year were sought in PubMed and PubMed Central with multiple keyword and acronym strategies. One verbatim strategy per lineage was written afterwards, re-run and archived; the records it returned that had not already been assessed were screened on title and publication type, and those that survived were assessed in duplicate. For every index trial published from 2016 through 2025 with at least one exchangeable earlier trial, the synthesis of all trials published before it was compared with the synthesis including it. The outcome was the proportion of index trials that changed a clinical decision category, defined on the risk-ratio scale with a region of no clinically important difference of 0.90 to 1.11. Categories and thresholds were fixed in an analysis plan frozen before extraction, which was not publicly registered; the interval method used in the main analysis was changed after the first run, and the frozen method is reported alongside. Study selection and data extraction were performed independently by two reviewers, with disagreements resolved by discussion and consensus, and every extracted count was verified against the source record. Results. Of 200 records assessed, 93 were eligible randomized trials and 42 reported the lineage outcome in a form recoverable per treatment group, so 41 trials entered five syntheses covering 11,817 analyzed patients. One lineage could not be analyzed at all. Twenty-nine index trials qualified. Depending on the interval method and the decision threshold, 0 to 1 of 29 index trials changed the decision category, and no trial did so in every specification in which it was included; in the amended main analysis the figure was 1 of 29 (3.4%; 95% CI, 0.6 to 17.2). Twenty-two index trials (76%), covering 8,207 analyzed patients, produced neither a category change nor a 20% gain in precision, and four trials reduced precision. Among the 14 index trials with retrievable full text, 8 cited a prior systematic review, 5 used one to justify the trial, and none presented an updated quantitative synthesis including its own result. Conclusions. Evidence deltas were reconstructable, and decision-category changes were rare and sensitive to analytic specification. The larger obstacle was reporting: 55% of eligible randomized trials could not enter such an accounting at all, because the outcome was reported without recoverable per-arm counts, reported only in another form, not measured, or reported with counts and percentages that could not be reconciled. This purposive, single-database pilot with lineage selection by the first author estimates feasibility rather than prevalence.