jazzPanda: A hybrid approach to find spatial markergenes in imaging-based spatial transcriptomics data

Spatial transcriptomics enables the understanding of the spatial architecture of tissues, providing deeper insight into tissue structure and cellular neighbourhoods. A crucial step in the analysis of spatial data is cell type identification. In single cell RNA-sequencing (scRNA-seq) analysis, cells are clustered according to their transcriptional similarity, and marker genes for each cluster identified. Marker analysis identifies genes highly expressed in each cluster compared to the remaining clusters, and these marker genes are used to annotate clusters with cell types. For spatial data, there are limited software tools for appropriate marker gene detection methods that account for the spatial distribution of gene expression. Tools developed for scRNA-seq ignore spatial information for the cells and genes. We have developed a hybrid approach to prioritize marker genes that uses the spatial coordinates of gene detections and cells making up clusters. We propose a binning approach that effectively "pseudobulks" gene detections and cells within clusters that can then be used as input into linear models for marker analysis. Our approach can account for multiple samples and background noise. We have tested our methods on several public datasets from different platforms including Xenium, CosMx and MERSCOPE. The marker genes detected by our method show strong spatial correlation with the corresponding clusters and have increased specificity compared to other methods. The method is implemented in the jazzPanda R Bioconductor package and is publicly available (https://bioconductor.org/packages/jazzPanda).