In time-lapse fluorescence imaging, single-particle-tracking is a powerful tool to monitor the dynamics of objects of interest, and extract information about biological processes. However, tracked particles can be subject to occlusion and intermittent detectability. When these phenomena persist over a few frames, tracking algorithms tend to produce multiple tracklets for the same particle. In this work, we introduce self-supervised learning of visual features to compare tracked particles, and we exploit both visual and positional distances to robustly stitch tracklets representing the same particle. We demonstrate the performance of our stitching framework on time-lapse fluorescence sequences of Hydra vulgaris neurons. Results show high stitching precision, and reduction of errors made by previous algorithms on the same data by a factor of two.
We introduce S3T (Self-Supervised Self-Distillation over Time), which, to the best of our knowledge, is the first fully self-contained framework for continuous video state tracking. Our method treats temporal sampling density as privileged information, based on the hypothesis that a denser view of the same clip recovers the running state more accurately. This view serves as the teacher, while a sparse-view student with the same weights learns to match its next-token distribution. The model generates its own target, so training requires no labels, separate teacher, or reward signal, and adds no inference cost. On LLaVA-OneVision-2-8B, S3T improves VSTAT accuracy by +1.74 as a single model, +2.38 with souping, and +2.70 with additional vision-encoder adaptation, while prior self-evolving methods leave state tracking largely unchanged. The capability learned from unlabeled synthetic clips transfers to real videos, improving performance by +7.95 on VSTAT-YouTube state-tracking questions and +4.50 on MVBench Action Count.
Shravan Venkatraman, Wenshuai Zhao, Mohammad Hassan Vali +1
We propose Physics-Informed Tracking (PIT), a video-based framework for tracking a single particle from video, where a neural network autoencoder localizes a particle as a heatmap peak (landmark) and a differentiable physics module embedded in the autoencoder constrains several landmarks over time (a trajectory) to satisfy known dynamics. The novel Physics-Informed Landmark Loss (PILL) compares this predicted trajectory back against the landmarks, enforcing physical consistency without labels. Its supervised variant (PILLS) instead compares the prediction against ground-truth position, velocity, and bounce from simulation, enabling end-to-end backpropagation. To support supervised and unsupervised learning, we use an autoencoder with a split bottleneck that separates A) tracking-related structure via landmark heatmaps from B) background noise and subsequent image reconstruction. We evaluate a replicated 26 factorial design (n = 4 replicates, 64 configurations), showing that PILLS consistently achieves sub-pixel tracking accuracy for the bilinear and physics-refined decoder outputs under both clean and noisy conditions.
Background and Objective: Quantitative analysis of cell dynamics is central to modern biological research, providing critical insights into immune cell interactions, disease progression, and drug mechanisms. Automated cell tracking in time-lapse microscopy remains challenging due to noise, morphological variations, overlapping cells, and dynamic events such as divisions and fusions. Methods: We present ARGUS, a framework for Accelerated, Robust, General, and Unsupervised Cell Tracking Solutions. ARGUS combines adaptive cell detection, dense Farneback optical-flow prediction, frame-to-frame linear assignment, and a sequence-level tracklet-refinement step that reconnects trajectory fragments across short temporal gaps. Results: On publicly available Cell Tracking Challenge datasets, ARGUS achieved detection accuracy of 0.905-0.971 and tracking accuracy of 0.897-0.964, with runtimes within 1 minute (5-6 seconds for 3 frames). Conclusions: ARGUS is a modular, interpretable framework that can be adapted to different imaging modalities and biological applications without training data or GPU infrastructure. The implementation is publicly available at https://github.com/Gitinc/argus