Uncertainty-Gated Min-Cost Flows for In Vivo NanoScale Synaptic Plasticity Tracking

Synapses are the fundamental unit of neural connectivity. They exhibit dynamic functional and structural changes that enable the brain to learn, adapt, and form memories. Recent advances in endogenous protein fluorescent labeling offer an opportunity to image synaptic strength in vivo and thus study the mechanisms underlying adaptive neural computation in living mice. Studying synaptic dynamics requires tracking individual signals of small, densely packed synapses over days while they change in size, position, and intensity between imaging sessions, and may even appear/disappear entirely. Tracking >100,000 dynamic, submicrometer particles is difficult even for state-of-the-art algorithms. Moreover, most algorithms rely on an isotropic uncertainty ball, assigning equal weight to the lateral plane (XY) and to the noisier axial dimension (Z), leading to poorer performance. To address these challenges and accurately track synapses in vivo, we developed SynTrack. We formulated SynTrack as a Maximum A Posteriori estimation problem under the anisotropic uncertainty ball, along with a fully temporally connected spatio-temporal graph to overcome long-term occlusions. SynTrack achieves a mean track length of 0.51 {micro}m with a Multiple Object Tracking Accuracy (MOTA) score of 88.8%, on par with MOTA scores of expert annotators but with massively increased speed and scalability. Over two weeks, we successfully track 65,000 synapses in 5.6 out of 8 imaging sessions on average, with 20,000 synapses being tracked in at least seven sessions. We present SynTrack as a state-of-the-art algorithm capable of high-resolution and fidelity tracking of synapse dynamics in behaving mice with unprecedented detail.