A variational sparse Gaussian-process method for detecting spatially variable genes and cellular interactions from spatial transcriptomics

Advanced spatially resolved transcriptomic (SRT) technologies preserve the spatial context of gene expression within tissues, enabling the study of context-dependent transcriptional regulation. Here, we propose VISGP, a variational sparse gaussian-process method for spatial variable genes (SVGs) and cellular interactions analysis from such data. VISGP utilizes variational inference and a sparse Gaussian process approximation, which efficiently models the posterior distribution with a set of inducing variables, thereby minimizing computational and memory complexity. When applied to simulated data and four real data sets, VISGP successfully identified the most SVGs than existing methods, and detected 85 spatially constrained ligand-receptor pairs that are missed by other methods. Together, VISGP provides a powerful strategy for decoding spatial gene regulation and cellular interactions, offering valuable biological insights into cellular heterogeneity and cancer pathology. TeaserA statistical framework that uncovers spatially variable genes and cell-cell communication, revealing hidden biological architecture in tissues.