Reliability and accuracy of medication-based performance measures for assessment of outpatient cardiology clinicians

Background: Multiple performance measures are used to assess the management of common cardiovascular conditions, such as coronary artery disease (CAD), heart failure with reduced ejection fraction (HFrEF), and atrial fibrillation (AF). These measures are sometimes used to assess individual clinicians, despite lack of evidence of reliability and accuracy. We used national Veteran Affairs (VA) data to test medication-based performance measures for assessment of clinician performance in outpatient cardiology care. Methods: This retrospective study examined VA cardiology clinician/Veteran dyads from 2017-2023 to evaluate 10 medication-based performance measures for CAD, HFrEF, or AF with elevated stroke risk. Achievement of each performance measure was defined as an active prescription of an indicated medication for each eligible patient encounter, summarized at the clinician level. Reliability was calculated using the beta-binomial method. A Bayesian causal model was used to assess the impact of case complexity on performance measure achievement. Finally, we performed peer review of a subset of cases (n=952) to assess the accuracy of three representative performance measures. Results: Of 456,071 unique cardiology encounters, 312,331 included CAD, 224,599 included HFrEF, and 178,678 included AF. These patients were treated by 1,043 cardiology clinicians. Median performance measure achievement ranged from 0.23 (SGLT2i for HFrEF) to 0.99 (beta blocker for HFrEF). Measure reliability varied from 0.63 to 0.90. Case complexity did not significantly affect performance measure achievement. Using peer review as the gold standard, accuracy was poor to moderate (balanced accuracy 0.51-0.73), and measures underestimated performance relative to peer review. Conclusions: Ten medication-based performance measures for CAD, HFrEF, and AF can be efficiently measured using electronic health record data, and most have sufficient reliability to support clinician benchmarking and the identification of potential care gaps. However, suboptimal accuracy compared with a peer review indicates that these measures may be insufficient as a standalone assessment of clinician quality.