Back

Quantifying Fit-for-Purpose in Real World Data: Data Grading and FitQ Scores

Jackson, M. L.; Manickam, R.; Derieg, D.; Gombar, S.; Low, Y. S.

2024-02-05 health informatics
10.1101/2024.02.02.24302239 medRxiv
Show abstract

Real-world evidence (RWE), derived from analysis of RWD, is increasingly used to guide decisions in drug development, regulatory oversight, and clinical decision-making. Evaluating the fitness-for-purpose of RWD sources is one key component to generating transparent RWE. Here, we demonstrate tools that fill two gaps in the data grading literature. These are the need for quantitative data grading scores, and the need for scoring mechanisms that can be run in automated fashion and at scale. The Real World Data Score (RWDS) rates the overall quality and completeness of a RWD source across a range of customizable metrics. The Fitness Quotient (FitQ) grades how well a specific data source fits a specific RWE query. In concert, these tools give producers and consumers of RWE evidence to assess the quality of the underlying RWD.

Matching journals

The top 2 journals account for 50% of the predicted probability mass.

50% of probability mass above

"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.