What Embedding-Based Guideline-Literature Drift Measures (and What It Doesn't): A Proof-of-Concept and Mechanistic Dissection of TIDE in Orthopedic Surgery
Grames, C.; Zakarian, P.; Franquemont, C.; Cabrera, A.; Elsissy, J.
Show abstract
Purpose: We introduce the Temporal Index of Divergent Evidence (TIDE), a metric that evaluates how the topic center of gravity of a corresponding corpus of literature aligns with, or shifts relative to, a recommendation's semantics over time. Methods: TIDE computes annual mean cosine similarity using MedCPT embedding vectors between a recommendation and corresponding PubMed records binned by year (2000-2024), testing for monotonic temporal trend (Kendall {tau}). Three recommendations with expected divergent, stable, and convergent trajectories were examined: two-stage exchange for periprosthetic joint infection (PJI), emergent fasciotomy for acute compartment syndrome (ACS), and non-operative management of distal radius fractures (DRF) in adults [≥] 65 years. The signal was characterized by an additive per-token attribution, an isolated vs. in-context analysis, and an adversarial minimal-pair test. Results: PJI diverged ({tau} = -0.84, year-shuffle p = 0.001), DRF converged ({tau} = +0.42, p = 0.005), and ACS showed no detectable trend ({tau} = -0.07, p = 0.66). The two significant directions (PJI, DRF) held under matched input [CLS] pooling, while ACS remained near zero. Publication volume baselines did not distinguish the three trajectories (publication count was universally positive, {tau} = +0.77 to +0.97). A reversed-meaning minimal-pair control showed little cosine sensitivity to reversal. Paper-level TF-IDF recovered the qualitative three-case pattern. Conclusion: Embedding-based drift analysis is a proof-of-concept candidate pre-triage signal that tracks topical alignment, not evidentiary direction; superiority over TF-IDF remains unproven. The present study is hypothesis-generating, showing potential use of embedding representations as an analytical unit rather than simply for retrieval.
Matching journals
The top 7 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Prospective Registration of Trials: Where we are, why, and how we could get better 91%
- Use and appropriateness of Reporting Guidelines in physical therapy research: a protocol for a meta-research study 90%
- Estimating and Testing an Index of Bias Attributable to Composite Outcomes in Comparative Studies 90%
Similar papers in this journal
- Creating an Indexing Scheme for Case Series Articles 91%
- Evaluation of SURUS: a Named Entity Recognition System to Extract Knowledge from Interventional Study Records 91%
- Investigator-initiated versus industry-sponsored trials – Visibility and relevance of randomized controlled trials in clinical practice guidelines (IMPACT) 91%
Similar papers in this journal
- Introducing the EMPIRE Index: A novel, value-based metric framework to measure the impact of medical publications 93%
- COVID-19-related research data availability and quality according to the FAIR principles: A meta-research study 91%
- Relationships among Commercial Biases and Author Conflicts of Interest in Biomedical Publishing 91%
Similar papers in this journal
- GPT for RCTs?: Using AI to measure adherence to reporting guidelines 92%
- Publishing at any cost: a cross-sectional study of the amount that medical researchers spend on open-access publishing each year 92%
- Agreeability testing of AMSTAR-PF, a tool for quality appraisal of systematic reviews of prognostic factor studies 91%
Similar papers in this journal
- Comparing scientific abstracts generated by ChatGPT to original abstracts using an artificial intelligence output detector, plagiarism detector, and blinded human reviewers 91%
- Artificial Intelligence for Surgical Scene Understanding: A Systematic Review and Reporting Quality Meta-Analysis 90%
- Bridging the Literacy Gap for Surgical Consents: An AI-Human Expert Collaborative Approach 89%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.