Why Risk Factors Minimally Change the ROC Curve AUC

Novel risk factors that improve statistical measures of fit on addition to established clinical prediction models often minimally change measures of discrimination (ROC curve AUC or c-index). As a result, measures of discrimination have been suggested to be insensitive in the evaluation of such models. To understand this phenomenon, it is necessary to focus on the population risk distributions produced by models with and without the risk factor. This is because these risk distribution fully determines the risk distributions of cases/patients and controls/nonpatients, which in turn fully determine the ROC curve and its AUC. Broader population risk distributions result in larger ROC curve AUCs. A fully independent risk factor with a relative risk of 2 added to the standard cardiovascular risk model produces risk distributions of those with or without the risk that are clearly different (which is evaluated by statistical measures of fit), while minimally broadening the population risk distributions (which is evaluated by measures of discrimination). The reason for this is that although addition of the risk factor replaces every risk stratum with higher and lower risk strata, this depopulated risk stratum is largely repopulated by similar splitting in neighboring risk strata. The interweaving of the the up and down migration paths to and from every point on the risk distribution results in a largely compensatory shuffling of risk assignments with minimal changes in the ROC curve AUC.