Turning an object into a scene: buildings activate scene-selective visual cortex independently of visual features

Human visual cortex contains regions that selectively respond to both scenes and large objects, particularly buildings. The cortical overlap between buildings and scenes has been attributed to shared visual features (e.g., cardinal orientations, rectilinearity). Alternative accounts propose that buildings may also activate scene representations indirectly, independently of specific visual features, for example because buildings evoke a sense of space. Here, we tested for such feature-independent activation by comparing EEG and fMRI responses in human participants (both sexes) to buildings and visually-matched boxes, and relating these responses to scene-selective responses. Buildings and boxes were matched, across exemplars, using image-based metrics, deep neural networks, and a perceptual similarity task. Time-resolved EEG decoding showed that buildings and boxes evoked discriminable responses from 360ms post-stimulus onset, incompatible with feedforward visual feature processing. Importantly, the building-box classifier generalized to discriminate scenes from chairs, providing EEG evidence for a representational overlap between buildings and scenes. Temporal generalization analyses further showed that the late building-selective response corresponded to an earlier scene-selective response, with a temporal offset of [~]130ms. Finally, ultra-fast fMRI (TR=140ms) revealed that these findings were mirrored in the response of the scene-selective parahippocampal place area (PPA), which similarly showed a feature-independent building-selective response that was delayed and prolonged relative to the scene-selective response. These results clarify the nature of building selectivity in visual cortex by showing that this selectivity can arise independently of visual features, putatively reflecting associative processes between buildings and scenes (or space). Significance StatementThe human ventral stream contains regions responding selectively to behaviorally-relevant categories such as places, faces, and words. Contrasting views attribute this selectivity either to the feedforward processing of visual features or to top-down input from domain-specific networks. Here, we show that neural selectivity for buildings persists in the absence of category-diagnostic visual features. EEG decoding showed that this high-level building response emerged late during object processing and corresponded to an earlier scene-selective response. Ultra-fast fMRI revealed that these findings were mirrored in the response of the scene-selective parahippocampal place area (PPA). These results show that selectivity for some categories can be decoupled from visual feature processing, reflecting associative processes that follow object recognition.