Benchmarking clustering, alignment, and integration methods for spatial transcriptomics

Spatial transcriptomics (ST) is advancing our understanding of complex tissues and organisms. However, building a robust clustering algorithm to define spatially coherent regions in a single tissue slice, and aligning or integrating multiple tissue slices originating from diverse sources for essential downstream analyses remain challenging. Numerous clustering, alignment, and integration methods have been specifically designed for ST data by leveraging its spatial information. The absence of benchmark studies complicates the selection of methods and future method development. Here we systematically benchmark a variety of state-of-the-art algorithms with a wide range of real and simulated datasets of varying sizes, technologies, species, and complexity. Different experimental metrics and analyses, like adjusted rand index (ARI), uniform manifold approximation and projection (UMAP) visualization, layer-wise and spot-to-spot alignment accuracy, spatial coherence score (SCS), and 3D reconstruction, are meticulously designed to assess method performance as well as data quality. We analyze the strengths and weaknesses of each method using diverse quantitative and qualitative metrics. This analysis leads to a comprehensive recommendation that covers multiple aspects for users. The code used for evaluation is available on GitHub. Additionally, we provide jupyter notebook tutorials and documentation to facilitate the reproduction of all benchmarking results and to support the study of new methods and new datasets (https://benchmarkst-reproducibility.readthedocs.io/en/latest/).