Graph neural networks best guide phenotypic virtual screening on cancer cell lines

Artificial intelligence is increasingly driving early drug design, offering novel approaches to virtual screening. Phenotypic virtual screening (PVS) aims to predict how cancer cell lines respond to different compounds by focusing on observable characteristics rather than specific molecular targets. Some studies have suggested that deep learning may not be the best approach for PVS. However, these studies are limited by the small number of tested molecules as well as not employing suitable performance metrics and dissimilar-molecules splits better mimicking the challenging chemical diversity of real-world screening libraries. Here we prepared 60 datasets, each containing approximately 30,000 to 50000 molecules tested for their growth inhibitory activities on one of the NCI-60 cancer cell lines. We evaluated the performance of five machine learning algorithms for PVS on these 60 problem instances. To provide a comprehensive evaluation, we employed two model validation types: the random split and the dissimilar-molecules split. The models were primarily evaluated using hit rate, a more suitable metric in VS contexts. The results show that all models are more challenged by test molecules that are substantially different from those in the training data. In both validation types, the D-MPNN algorithm, a graph-based deep neural network, was found to be the most suitable for building predictive models for this PVS problem.