Benchmarking Artificial Intelligence Models for Predicting Nuclear Receptor Activity from Tox21 Assays

Tox21 assays compile extensive chemical bioactivity data across diverse biological targets, making them widely utilized resources for in silico model development. Nuclear receptor-specific assays within this dataset are particularly valuable for screening potential endocrine disrupting chemicals. This study presents a comprehensive benchmarking of diverse machine learning (ML), deep learning (DL), and transformer-based architectures with varied chemical feature representations across nuclear receptor assays. First, 43 datasets associated with 18 nuclear receptors within Tox21 assays were systematically curated from ToxCast invitrodb v4.3. Upon testing across these datasets, model performance was found to be dependent on the degree of class imbalance. Tree-based ML models such as random forest (RF) and extreme gradient boosting (XGBoost) trained on descriptors, or combination of descriptors and fingerprints, consistently outperformed in datasets with higher proportions of active chemicals (>10%), while DL models showed greater robustness for those with moderate proportions (5-10%). Further analysis revealed that approximately 40% of misclassified active chemicals occupied structurally isolated regions of the chemical space, suggesting absence of close structural analogues in the training set potentially contributed to their misclassification. External validation using in vitro and in vivo androgen and estrogen receptor bioactivity data showed generally good concordance. Finally, a systematic literature review revealed that the models in this study span wider range of architectures, feature representations, and assay endpoints, and are broadly comparable to or better than existing work. Overall, insights from this study can inform the development of more reliable in silico tools supporting new approach methodologies for nuclear receptor bioactivity predictions.