Framed RSA: Representational comparisons that honor bothgeometry and population-mean response preferences

Representational similarity analysis (RSA) characterizes the geometry of neural activity patterns elicited by different stimuli while discarding information about neural response preferences, regional population-mean activity and the absolute location and orientation of the patterns in the multivariate response space. Analyses of the tuning of individual neurons or voxels and of the population-mean activation are often performed separately from RSA and thought of as serving complementary purposes. When evaluating alternative representational models, invariance to certain aspects of the neural code is desirable because systems might use superficially different encodings to implement the same computations. However, neural preferences and regional-mean activation are arguably physiologically and mechanistically important, and so we may want for our models to predict them correctly. Here we introduce a novel analysis technique, framed RSA, which honors both the geometry and the population-mean preferences in evaluating model-predicted representations. To achieve this, we augment the set of patterns that define the geometry by two reference patterns: the zero-point (origin) and a uniform constant pattern in the multivariate response space, enabling RSA to incorporate information about the global location, orientation, and mean activation of neural population codes. First we present the mathematical and methodological underpinnings of framed RSA, including how it interacts with different RSA analysis choices, such as the use of cross-validated dissimilarity estimates and whitened RDM comparators. Second, we show empirically that framed RSA generally improves accuracy for both brain-region identification (using fMRI data from the Natural Scenes Dataset and the Things Ventral Stream Spiking Dataset) and deep-neural-network-layer identification relative to existing RSA approaches. By incorporating neural population preferences into model evaluation, framed RSA enables more mechanistically meaningful model comparisons and benefits from improved power for model-comparative inference.