Critical issues found in "Dissecting cell identity via network inference and in silico gene perturbation"

1In the 2023 Nature publication "Dissecting cell identity via network inference and in silico gene perturbation" [1], the authors introduced CellOracle (CO), a novel method leveraging mRNA-seq and ATAC-seq data to construct gene regulatory networks (GRNs), which are subsequently used for gene perturbation. They designed CO to account for the role of distal cis-regulatory elements, e.g. enhancers, as well as proximal promoters in the gene regulation system. For this purpose, they employed Cicero to determine the co-accessibility scores between peaks, provided by ATAC-seq data. These scores are then used to identify the interaction of distal regions with the target gene. Using CO, they have conducted multiple perturbation studies on different organisms and identified novel phenotypes resulting from transcriptional factor (TF) perturbation. In addition, they benchmarked COs performance using ChIP-seq data as ground truth against other state-of-the-art GRN methods across multiple mouse tissue samples. However, our evaluation reveals critical limitations in the implementation of their methodology, both in terms of ATAC-seq data integration as well as benchmarking. In this report, we first explain the limitations in their approach of integrating ATAC-seq data. We show that the proposed algorithm fails to account for distal regulatory interactions. After, we present the issues associated with their benchmarking algorithm and the data used for benchmarking. We show that their findings regarding the comparative performance of CO against other GRN inference methods is invalid and requires further evaluation. In conclusion, we detect multiple inaccuracies in this paper which undermine the validity of their published protocol and the results. The materials supporting our findings are accessible on GitHub1.