RGAST: Relational Graph Attention Network for Spatial Transcriptome Analysis

Cell-cell communication (CCC) plays a fundamental role in tissue organization and function. Recent advances in spatial transcriptomics (ST) technologies have enabled high-resolution mapping of CCC at single-cell level. However, existing computational approaches for CCC inference face several limitations, including reliance on predefined ligand-receptor databases, loss of single-cell resolution, and inability to model long range communications. To address these challenges, we developed RGAST, a deep learning framework that integrates both spatial proximity and transcriptional profiles to reconstruct multi-scale CCC networks de novo. In our analysis, RGAST revealed directional communication from peripheral to central nuclei in mouse hypothalamus and tumor invasion signaling axis in breast cancer. Leveraging a relational graph attention network, RGAST effectively captures both local and global communication patterns while learning low-dimensional representations of ST data, which are versatile in multiple downstream tasks. Our results demonstrate that RGAST enhances spatial domain identification accuracy by approximately 10% compared to the second method in 10X Visium DLPFC dataset. Furthermore, RGAST facilitates the discovery of spatially variable genes, enables more precise cell trajectory inference and reveals intricate 3D spatial patterns across multiple sections of ST data.