Widespread data leakage inflates performance estimates in cancer drug response prediction
Asiaee, A.; Strauch, J.; Azinfar, L.; Pal, S.; Pua, H. H.; Long, J. P.; Coombes, K. R.
Show abstract
Drug response prediction models are widely used to nominate biomarkers and guide preclinical drug prioritization. However, their reported performance hinges on rigorous separation of training and test data during cross-validation (CV). Here we show that a commonly used pattern--supervised feature screening performed on the full dataset before CV--introduces data leakage that systematically underestimates prediction error. Analyzing 265 drugs across 1,462 cancer cell lines, we find that leakage-free CV increases mean squared error (MSE) by 16.6%, with low feature-set overlap between leaked and leakage-free pipelines (mean Jaccard 0.18). A manual audit of 12 recent deep learning and classical methods found confirmed leakage in 10. Such inflated performance estimates likely contribute to computational predictions that fail during independent validation or experimental follow-up. We provide an audit guide and reference implementation to prevent leakage, and introduce a tissue-aware Data Shared Elastic Net (DSEN) that, under correct evaluation, improves prediction for 65.7% of drugs while yielding sparser, more targeted biomarker sets.
Matching journals
The top 10 journals account for 50% of the predicted probability mass.