S3KF: Spherical State-Space Kalman Filtering for Panoramic 3D Multi-Object Tracking

Multi-Object Tracking (MOT) is essential for embodied perception, where panoramic cameras provide a 360° field of view to reduce blind spots and keep surrounding targets observable for longer durations. However, panoramic MOT requires motion estimates and association cues consistent with its imaging geometry. In videos represented in equirectangular projection (ERP), horizontal coordinates wrap at the image seam, while frame-wise relative-depth estimates may fluctuate over time. These inconsistencies can disrupt trajectory prediction and identity association. To address these challenges, we propose CylindTrack, a depth-aware cylindrical tracking-by-detection framework that integrates continuous angular motion modeling with temporally filtered relative-depth cues. Specifically, Depth-Temporal Trajectory Modeling (DTM) promotes instance depth from an isolated frame-wise cue into temporally filtered trajectory-level states. Spherical Spatio-Temporal Consistency Learning (SSTC) combines a Temporal Mixer and Spherical Geometry-aware Attention to enhance temporal coherence and panoramic geometric alignment in depth-aware representations. Furthermore, a Topology-Aware Cylindrical Motion Model (TCMM) lifts horizontal motion into a continuous angular state space, enabling seam-consistent motion prediction and association. By jointly modeling trajectory-level depth consistency and panoramic topology, CylindTrack improves identity preservation and trajectory continuity in challenging panoramic scenes. Experiments on QuadTrack and JRDB demonstrate improvements in identity-oriented metrics with practical efficiency, achieving HOTA scores of 33.67 and 31.12, IDF1 scores of 40.45 and 34.33, and tracking speeds of 28.56 and 21.34 FPS, respectively. The source code will be released at https://github.com/warriordby/CylindTrack.

Monocular videos record 3D scenes as sequences of 2D image-plane projections, obscuring depth and spatial relationships. Multi-object trackers localize and associate objects primarily using appearance and geometry observed only in the image plane, inheriting these ambiguities. To address this limitation, we introduce PLANET, an end-to-end multi-object tracker designed to move beyond the image plane. As an enabling step, we lift existing 2D tracking datasets into 3D. We then form world-grounded queries by embedding reconstructed 3D scene geometry into the features and positional encodings used during query formation. An auxiliary 3D location prediction task further encourages the queries to encode object positions during training. A complementary dual-resolution temporal memory preserves this evidence across longer temporal gaps. As a result, PLANET achieves state-of-the-art performance across three diverse benchmarks.

Panoramic sensing offers wide field-of-view coverage, yet 3D reconstruction from sparse panoramas remains challenging under rotation-dominant, weak-parallax motion. In such regimes, SfM/SLAM initialization is often ill-conditioned and unreliable. We present PanoImager, an SfM-free framework that combines feed-forward pose/depth priors, geometry-conditioned diffusion view completion, and depth-guided 3DGS optimization. Given only a few panoramic images, PanoImager decomposes them into local perspective views, synthesizes auxiliary observations to enrich sparse evidence, and stabilizes Gaussian optimization for improved cross-view consistency. Experiments on multiple benchmarks show improved stability under extreme sparsity, suggesting PanoImager as an offline/background component for map refinement when SfM/SLAM fails to initialize.