Trust-Aware Sequence-to-Function Modelling in Regulatory Genomics
Onawole, A.; Basiru, S.; Sanni, M. O.; Aiyedun, M.; Sulaimon, R.
Show abstract
Objective: Sequence-to-function models increasingly predict regulatory activity, such as chromatin accessibility, directly from DNA sequence, and are used to interpret non-coding genetic variation. Standard accuracy metrics, computed over a held-out set of genomic regions, do not establish whether an individual prediction remains reliable once the input sequence departs from that set, nor whether a model's attribution-based explanation is biologically grounded rather than coincidental. We develop and evaluate RegTrust-XAI, a trust-aware framework separating these questions using three inference-time signals: ensemble consensus, motif-grounded attribution coherence, and applicability-domain distance. Methods: A five-model convolutional ensemble was trained on 517,790 K562 ATAC-seq windows and evaluated on a held-out chromosome test set (chr8/chr9, n = 42,844). Consensus, coherence, and applicability-domain distance were each tested against prediction error, alongside complementary sequence-novelty analyses and validation against an independent lentiMPRA reporter assay and saturation-mutagenesis MPRA data at the PKLR promoter. Results: The ensemble reached Spearman {rho} = 0.782, with skill of 0.328 over a constant-value null predictor. High-consensus predictions (Scenarios A+B) were consistently enriched for lower error than low-consensus predictions (Scenarios C+D), and attribution coherence further separated error within the high-consensus population (mean absolute error 0.396 versus 0.435, p = 9.6e-10). Applicability-domain distance showed a monotonic error gradient across six distance bands. A 4-mer composition-divergence metric was negatively associated with error and anti-correlated with applicability-domain distance, so composition-based and model-relevant novelty are not equivalent. Attribution transfer to lentiMPRA was assay- and subgroup-dependent, and predicted allele-substitution effects correlated with measured saturation-mutagenesis effects at the PKLR promoter at both 24 h and 48 h ({rho} = 0.227 and 0.235). Motif-specific perturbation further showed that regulatory attributions were strongly context-dependent, with more than 90% of multi-instance motif modules exhibiting superadditive joint effects. Conclusions: Prediction reliability, explanation validity, and sequence novelty are related but distinct properties of a sequence-to-function model. Evaluating each explicitly gives a more complete basis for deciding when to act on a prediction than accuracy alone.
Matching journals
The top 6 journals account for 50% of the predicted probability mass.
Similar papers in this journal
- Comprehensive Evaluation of Diverse Massively Parallel Reporter Assays to Functionally Characterize Human Enhancers Genome-wide 93%
- EvoAug: improving generalization and interpretability of genomic deep neural networks with evolution-inspired data augmentations 92%
- Polygraph: A Software Framework for the Systematic Assessment of Synthetic Regulatory DNA Elements 92%
Similar papers in this journal
- Sequence-based chromatin activity modeling and regulatory impact prediction of genetic variants in farmed animals using deep learning 94%
- Transfer learning identifies sequence determinants of regulatory element accessibility 93%
- Towards Personalized Epigenomics: Learning Shared Chromatin Landscapes and Joint De-Noising of Histone Modification Assays 91%
Similar papers in this journal
- Gkmexplain: Fast and Accurate Interpretation of Nonlinear Gapped k-mer Support Vector Machines Using Integrated Gradients 94%
- DECODE: A Deep-learning Framework for Condensing Enhancers and Refining Boundaries with Large-scale Functional Assays 92%
- Chicdiff: a computational pipeline for detecting differential chromosomal interactions in Capture Hi-C data 92%
Similar papers in this journal
- Genome-wide Enhancer Maps Differ Significantly in Genomic Distribution, Evolution, and Function 93%
- Computational Chromosome Conformation Capture by Correlation of ChIP-seq at CTCF motifs 91%
- Systematic benchmark of state-of-the-art variant calling pipelines identifies major factors affecting accuracy of coding sequence variant discovery 90%
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.