Documented Pain Relief After Emergency Department Headache Treatment Is Not a Stable Outcome: Reassessment Timing, Missingness, and Score Selection

Background: Whether a patient's pain improved after emergency department (ED) treatment is read from the record to benchmark EDs, compare drugs, and label research outcomes. It is interpretable only if a post-treatment score is recorded, appropriately timed, and chosen by a fixed rule; its stability across these choices is unknown. Methods: Retrospective measurement study of adult headache visits in a de-identified ED database (MIMIC-IV-ED, 2011-2019). Among treated visits, we quantified reassessment completeness by time window, estimated meaningful relief (a reduction of at least 2 points) under score-selection rules and missing-data assumptions, tested whether reassessment was predictable at treatment, and compared headache with other painful presentations. Results: Among 19,501 visits (15,273 patients), 13,682 (70.2%) were treated. A post-treatment pain score appeared at any time for 77.1% (95% CI, 76.4 to 77.8), but within 2 hours of the analgesic for only 47.9% and within 1 hour for 27.5%. Meaningful relief was 66.9% using the first post-treatment score but 81.0% and 83.4% using the last or lowest score; it was 67.5% under inverse-probability weighting and could be bounded only between 51.8% and 74.4%. Whether a score was recorded was weakly predictable at treatment (area under the curve, 0.566) and unrelated to baseline pain. Completeness was similar across headache strata and comparator painful presentations. In an independent ED (MC-MED, a different EHR), the score-selection effect replicated: relief rose from 71.1% (first) to 80.6% (last) and 83.4% (lowest). Conclusions: Documented pain relief after ED headache treatment was not a stable outcome: it varied with the reassessment window and score-selection rule, was not point-identified for unreassessed patients, and behaved like other painful ED presentations. Programs and research that use documented relief should prespecify the reassessment window, score-selection rule, completeness denominator, and a missing-data range, and favor protocol-timed reassessment.