Enhancing Scientific Validity by Integrating EHR and Claims Data in the National Clinical Cohort Collaborative

Purpose: Accurately capturing a patient's health journey is essential for ensuring the validity of scientific and public health studies. Electronic health records (EHR) and administrative claims are commonly used in healthcare research, albeit separately. Previous research has demonstrated that EHR and claims capture different aspects of patient history and care, potentially introducing bias when used alone. In this study, we leveraged the largest public US EHR resource, the National Clinical Cohort Collaborative (N3C), alongside Centers for Medicare & Medicaid Services (CMS) claims data to compare their information coverage and agreement. Methods: We integrated N3C EHR and CMS data to create one cohesive dataset and quantified the proportion of key variables present in EHR alone, CMS alone, and the overlap present in both data sources. Overlap in information coverage was quantified pre- and post-landmark diagnosis of COVID-19. To evaluate the impact of data coverage differences, we conducted a case study evaluating the effect of COVID-19 vaccination on ischemic stroke using both data sources alone and in combination. Results: CMS data helped identify many individuals that were duplicated among EHR identifiers. Examination of the overlap showed that CMS data more completely captured conditions, while EHR data more completely captured medications after COVID-19 diagnosis. Some variables exhibited marked differences over time after landmark COVID-19 diagnosis. Using target trial emulation and self-controlled case series methods, we found that data capture variations altered effect estimates: EHR data alone yielded lower risk of ischemic stroke after vaccination in the target trial emulation (odds ratio 0.47, 95% CI 0.28-0.79), with a smaller, non-significant reduction in the self-controlled case series (incidence rate ratio 0.81, 95% CI 0.64-1.02), whereas analyses using CMS data alone or both sources combined yielded estimates close to the null in both designs. Conclusions: Results indicated that when using either a national database combining multiple EHR sources, or CMS claims data alone, patients' medical histories are less completely captured compared to when these data sources are combined. Case study results highlighted the value of combining data sources for not only yielding fuller patient health histories and valid scientific results, but for identifying and explaining biases that may emerge in research studies that rely on a single data source.