Spatial transcriptomic data reveals pure cell types via the mosaic hypothesis

Neurons display remarkable diversity in their anatomical, molecular, and physiological properties. Although observed stereotypy in subsets of neurons is a pillar of neuroscience, clustering in high-dimensional feature spaces, such as those defined by single-cell RNA-seq data, is often inconclusive, with cells seemingly occupying continuous, rather than discrete, regions. In the retina, a layered structure, neurons of the same discrete type avoid spatial proximity with each other. While this principle, which is independent of clustering in feature space, has been a gold standard for retinal cell types, its applicability to the cortex has been only sparsely explored. Here, we provide evidence for such a mosaic hypothesis by developing a statistical point process analysis framework for spatial transcriptomic data. We demonstrate spatial avoidance across many excitatory and inhibitory neuronal types. Spatial avoidance disappears when cell types are merged, potentially offering a gold standard metric for evaluating the purity of putative cell types. Significance statementWhile morphologically or molecularly-defined cell types and their taxonomy are a pillar of modern neuroscience, the extent to which their discrete treatment reflects biology is far from settled. A functional hypothesis concerning cortical neuronal cell types with roots in retina research suggests an anatomical test for the existence and identification of pure, discrete cell types. We describe evidence for this decades-old hypothesis in the mouse neocortex and detail the associated computational methodology based on spatial point processes.