A comprehensive benchmark and guide for sequence-function interpretable deep learning models in genomics

The development of sequence-based deep learning methods has greatly increased our understanding of how sequence determines function. In parallel, numerous interpretable algorithms have been developed to address complex tasks, such as elucidating sequence regulatory syntax and analyzing non-coding variants from trained models. However, few studies have systematically compared and evaluated the performance and interpretability of these algorithms. Here, we introduce a comprehensive benchmark framework for evaluating sequence-to-function models. We systematically evaluated multiple models and DNA language foundation models using 369 ATAC-seq datasets, employing diverse training strategies and evaluation metrics to uncover their critical strengths and limitations. Our benchmark study highlights that different model architectures and interpretability methods are better suited to specific scenarios. Negative samples derived from naturally inactive regions outperform synthetic sequences, whereas single-cell tasks require specialized models. Additionally, we demonstrate that interpretable sequence-function models can complement traditional sequence alignment methods in studying cross-species enhancer regulatory logic. We also provide a pipeline to help researchers select the optimal sequence-function prediction and interpretability algorithms.