Rapid robust high-fidelity 3D neuronal extraction from multi-view projections

Recent developments in imaging facilitate large-scale three-dimensional (3D) neuronal recording. While these huge amounts of data shed new light on population-level neural coding, it is much harder to extract neuronal calcium dynamics from 3D volumes than 2D images with the existence of noise and scattering. Here, we presented DeepWonder3D, a general end-to- end pipeline for rapid and robust 3D neuronal extraction with high fidelity. Instead of processing voxel by voxel, DeepWonder3D works on the multi-view projections of 3D imaging data obtained either digitally or optically through specific point spread functions (PSFs) for general applicability to diverse techniques, including point-scanning microscopy, light- field microscopy, and two-photon synthetic aperture microscopy. Integrating denoising, resolution registration, background removal, neuronal extraction, and multi-view fusion into a unified pipeline tailored for large-scale high-resolution datasets contaminated by noise and scattering, DeepWonder3D outperforms state-of-the-art methods in 3D localization accuracy with a 10-fold reduction in computational costs, validated by numerical simulations and a hybrid two-photon/light-field imaging system. With the RUSH3D mesoscope, DeepWonder3D achieves rapid high-fidelity 3D calcium extraction of tens of thousands of neurons across the mouse cortex within hours.