Universal Transformer-Based Tracker for Accurate Tracking of Particles and Cells in Microscopy

Accurate tracking of subcellular structures and cells under microscopy supports heavily the studies of their dynamic processes. However, the complex motion and the similar appearances of objects pose significant challenges in accurately identifying the identical object across multiple detection results without introducing ambiguities in trajectory tracking. Here we propose a Universal Transformer-Based Tracker (UTT) that achieves the accurate tracking in various biologically relevant scenarios. The self-attention of Transformer extracts features and patterns from trajectories, and the cross-attention of Transformer captures matching relations between history tracks and future hypothesis tracklets (fragments of track). Our tracker uses the Transformer to handle the tracking of particles with diverse motion dynamics, and outperforms existing tracking algorithms in terms of accuracy. Furthermore,the tracker utilizes estimated particle positions to counteract missed detections, thereby improving tracking robustness under varying ratios of missing detections. We demonstrate the flexibility and versatility of our approach by adaptively integrating motion and appearance cues in cell tracking applications.