OmniGlimpse: Asymmetric Identity Memory for Online Multi-View Tracking

Rangya Zhang; Jiaping Xiao; Yujing Shang; Yuhang Zhang; Mingsheng Li; Mir Feroskhan

cs.CVOct 27, 2026

OmniGlimpse: Asymmetric Identity Memory for Online Multi-View Tracking

Authors: Rangya Zhang, Jiaping Xiao, Yujing Shang, Yuhang Zhang, Mingsheng Li, Mir Feroskhan

Organizations: 1Nanyang Technological University

Abstract

Multi-view tracking extends visual coverage beyond a single camera, providing complementary observations when targets are occluded or leave the view. This raises a question: how can observations from different views be matched to the correct identities without offline scene-specific training or camera calibration? Progress toward this goal remains limited because many offline formulations assume such deployment-specific knowledge, while online adaptation can amplify association errors. We study this problem as online multi-view identity maintenance (OMIM): targets are initialized once per view, with no further identity labels provided after each initialization. To this end, we introduce OmniGlimpse, an asymmetric propose-verify framework that separates adaptive association from stable identity consolidation. At the representation level, short-term local and cross-view evidence is mapped into target-specific adaptive spaces to propose associations, whereas long-term identity distributions are maintained for verification in a canonical space defined by a frozen encoder. At the update level, short-term evidence may guide current associations, whereas, after bootstrapping, only verified observations may update persistent memory or drive adaptation. Under a unified target-specified evaluation protocol, OmniGlimpse improves over the best evaluated online baseline for each metric by 9.1 IDF1 and 22.1 CVIDF1 points on M3Track. The corresponding gains on MvMHAT are 20.3 and 26.2 points. Despite using no offline scene-specific training, it outperforms offline-trained references in IDF1 and remains competitive in cross-view association. These results support an approach to online identity maintenance that adapts to changing appearances while consolidating reliable evidence into stable identity anchors.

Figures & tables

Figure 1: Overview of OmniGlimpse. Top: targets are initialized once per view and maintained through adaptive association, canonical verification, and verified updates. Bottom: OmniGlimpse supports cross-view recovery, appearance adaptation, and resistance to distractors.

Figure 2: Overview of OmniGlimpse. Short-term evidence supports association in adaptive spaces, while long-term distributions provide canonical-space verification. After initialization and bootstrapping, only admitted observations may update persistent memory and local adapters.

Method	HOTA $\uparrow$	DetA $\uparrow$	AssA $\uparrow$	IDF1 $\uparrow$	MOTA $\uparrow$	MOTP $\uparrow$	IDs $\downarrow$	MDA $\uparrow$	CVIDP $\uparrow$	CVIDR $\uparrow$	CVIDF1 $\uparrow$	CVMA $\uparrow$
Offline-trained MVMT references
MvMHAT	.5832	.4407	.7723	.6267	.4641	.9335	11	.1620	.2914	.1920	.2277	.3204
CrossMOT	.5970	.5540	.6445	.7945	.6628	.8182	116	.5635	.6150	.5499	.5786	.6335
AGW	.4748	.3928	.5773	.6347	.4630	.7936	9	.2350	.3374	.2556	.2864	.3717
Online baselines
OCLReID	.5862	.4810	.7173	.7424	.5795	.7972	18	.3290	.4480	.3552	.3918	.4586

Table 1: Selected-target results on M3Track. Offline references use target-selection information, including ground-truth-assisted trajectory selection for CrossMOT and AGW and full-sequence appearance matching for MvMHAT (Appendix B.2 ). Best and second-best in bold and underline.

Method	HOTA $\uparrow$	DetA $\uparrow$	AssA $\uparrow$	IDF1 $\uparrow$	MOTA $\uparrow$	MOTP $\uparrow$	IDs $\downarrow$	MDA $\uparrow$	CVIDP $\uparrow$	CVIDR $\uparrow$	CVIDF1 $\uparrow$	CVMA $\uparrow$
Offline-trained MVMT references
MvMHAT	.3646	.1786	.7443	.3012	.1448	.9556	1	.0500	.0522	.0505	.0513	.2354
CrossMOT	.5217	.4448	.6130	.6901	.4874	.8196	17	.5626	.6380	.5819	.6078	.7190
AGW	.5761	.4352	.7628	.5875	.3518	.9993	6	.3448	.4687	.3759	.4131	.5114
Online baselines
OCLReID	.4644	.3060	.7133	.5292	.3290	.8403	3	.2280	.2975	.2454	.2676	.4245

Table 2: Selected-target results on MvMHAT. Offline references use information for target selection: CrossMOT uses ground-truth-assisted trajectory selection; MvMHAT and AGW use full-sequence appearance matching (Appendix B.2 ). Best and second-best results are shown in bold and underline.

Variant	Rel.	Cross	Adapt.	Gauss.	V-Upd.	HOTA $\uparrow$	DetA $\uparrow$	IDF1 $\uparrow$	MOTA $\uparrow$	IDs $\downarrow$	MDA $\uparrow$	CVIDP $\uparrow$	CVIDF1 $\uparrow$	CVMA $\uparrow$
Base	–	–	–	–	–	.6240	.5548	.7999	.6801	19	.3995	.5602	.4881	.5395
+ Reliability Controller	$\surd$	–	–	–	–	.6386	.5802	.8246	.6989	46	.4769	.6271	.5608	.5805
+ Cross-view Assistance	$\surd$	$\surd$	–	–	–	.6389	.5890	.8280	.7100	45	.5360	.6460	.6000	.6250
+ Online Contrastive	$\surd$	$\surd$	$\surd$	–	–	.6528	.5964	.8453	.7322	83	.5255	.6271	.5850	.6398
Full w/o Rel.	–	$\surd$	$\surd$	$\surd$	$\surd$	.6178	.5482	.7938	.6710	32	.3924	.5507	.4796	.5333
w/o Online	$\surd$	$\surd$	–	$\surd$	$\surd$	.6250	.5650	.8050	.6900	26	.5140	.6210	.5710	.6210

Table 3: Ablation study on 19 non-dark M3Track test instances. The upper block shows incremental variants and the lower block full-model controls (Appendix B.3 ). V-Upd. denotes verification-gated updates; Adaptive Query uses adapted features. Best and second-best are in bold and underline.

Figure 3: Identity maintenance in the M3Track garden scene. Columns show synchronized observations; the first row shows the reference view. Yellow dashed boxes mark the target, and cyan boxes mark confirmed predictions.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Value
Detector / canonical backbone	YOLOX-L / ResNet-18
Canonical checkpoint source	OCLReID-trained ResNet-18
Number of part cues $K$	4
Global and part dimensions $D_{k}$	512
Detection score / NMS IoU threshold	0.60 / 0.35
Support aggregation $L$	3

Appendix

Table 4: Association, verification, and admission settings. Capacities count observations or prototypes, not individual feature coordinates.

Setting	Value
Adapter hidden width	128
Optimizer	Adam
Learning rate / weight decay	$10^{-4}$ / $10^{-4}$
Optimization interval per target–view pair	5 newly admitted local observations
Optimizer steps per eligible batch	1
Maximum / minimum triplets per batch	16 / 4

Appendix

Table 5: Local adapter learning and classifier settings. Ridge regularization refers to the class-normalized objective in Eq. 12 .

Figure 4: Additional comparison in the M3Track wood scene. Columns correspond to dataset frames 1861, 1867, 1871, and 1875. The reference view is shown once; the other rows use identical images and crops for the three configurations. Yellow dashed boxes denote the target and cyan boxes denote confirmed outputs; tentative candidates are not drawn. Displayed View 1 and View 2 correspond to source view2 and view1, respectively.

Multi-Object Tracking

Multi-object tracking (MOT) links detections across video frames to maintain object identities through motion, occlusion, and changing viewpoints.

See all 193 papers

Sep 16, 2026

Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries

Online multi camera 3D tracking must maintain scene global identities across synchronized views, yet query-based trackers carry these identities only implicitly in the instance bank, where they fragment upon query interruption. We present an online architecture that recovers association accuracy by predicting IDs explicitly over recurrent sparse queries. An outside-in Sparse4D detector fuses calibrated views into world frame 3D detections while propagating a sparse query bank, and a causal MOTIP ID decoder associates detections against a finite trajectory memory. We adapt MOTIP's relative-ID prediction and recycled slot runtime to globally fused 3D observations, and introduce metric spatial gating and proximity based newborn recovery. On the official 2026 AI City Challenge Track 1 test set, our method raises HOTA from 29.63 with native instance bank identities to 38.01, primarily through an AssA increase from 20.83 to 31.10, and ranks third on the public leaderboard. Full-sequence validation over all 9,000 frames of each scene shows that decoupled ID training improves HOTA over native identities, whereas continuing detector training alongside the detached ID objective produces scene-dependent gains and losses.

Pragyan Shrestha, Haruto Nakayama, Atom Scott

Playbox Inc., Tokyo, Japan

3D MOT Multi-Object Tracking

Ask PDF

Mar 16, 2026

ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation

Multi-View Multi-Object Tracking (MV-MOT) aims to localize and maintain consistent identities of objects observed by multiple sensors. This task is challenging, as viewpoint changes and occlusion disrupt identity consistency across views and time. Recent end-to-end approaches address this by jointly learning 2D Bird's Eye View (BEV) representations and identity associations, achieving high tracking accuracy. However, these methods offer no principled uncertainty accounting and remain tightly coupled to their training configuration, limiting generalization across sensor layouts, modalities, or datasets without retraining. We propose ModTrack, a modular MV-MOT system that matches end-to-end performance while providing cross-modal, sensor-agnostic generalization and traceable uncertainty. ModTrack confines learning methods to just the Detection and Feature Extraction stage of the MV-MOT pipeline, performing all fusion, association, and tracking with closed-form analytical methods. Our design reduces each sensor's output to calibrated position-covariance pairs

(\mathbf{z}, R)

; cross-view clustering and precision-weighted fusion then yield unified estimates

(\hat{\mathbf{z}}, \hat{R})

for identity assignment and temporal tracking. A feedback-coupled, identity-informed Gaussian Mixture Probability Hypothesis Density (GM-PHD) filter with HMM motion modes uses these fused estimates to maintain identities under missed detections and heavy occlusion. ModTrack achieves 95.5 IDF1 and 91.4 MOTA on WildTrack, surpassing all prior modular methods by over 21 points and rivaling the state-of-the-art end-to-end methods while providing deployment flexibility they cannot. Specifically, the same tracker core transfers unchanged to MultiviewX and RadarScenes, with only perception-module replacement required to extend to new domains and sensor modalities.

Aditya Iyer, Jack Roberts, Nora Ayanian

Department of Computer Science, Brown University, Providence, RI, USA · School of Engineering, Brown University, Providence, RI, USA

Multi-Object Tracking Uncertainty-Aware Multimodal Fusion Bayesian Filtering Multimodal Sensor Fusion

Ask PDF

Jul 26, 2026

Beyond Appearance: A Multi-cue Framework and Large-scale Benchmark for Pedestrian Association and Tracking on Mobile Aerial-Ground Platforms

Multi-view Multi-object Association and Tracking (MvMoAT) associates objects across camera views and tracks them over time, supporting identity persistence and forensic trajectory reconstruction in multi-platform cooperative perception. Unlike conventional multiple object tracking, MvMoAT faces frequent viewpoint shifts that distort appearance and undermine cross-view association and temporal tracking. We propose FUSION, a viewpoint-robust Feature Unification framework for multi-view aSsociation and IdentificatiON. Its Multi-cue Adaptive Combination (MAC) module adaptively integrates viewpoint-invariant cues with appearance features to improve cross-view association, while Online Multi-view Feature Synchronization (OMFS) aggregates pedestrian features across historical and cross-view frames for temporally consistent tracking. We also introduce RealMvMoAT, a large-scale benchmark featuring substantial inter- and intra-camera viewpoint variation. It contains 504.9K frames from 7 cameras (5 UAV and 2 ground views) across 10 scenes, with over 7.3M identity-labeled bounding boxes. All cameras exhibit random and substantial motion. To the best of our knowledge, RealMvMoAT is the largest MvMoAT dataset to date. Its scale, viewpoint diversity, complex platform motion, and realistic trajectories provide a comprehensive resource for future research. Experiments on RealMvMoAT and six public benchmarks show that FUSION achieves state-of-the-art performance.

Ruiqi Wu, Bingliang Jiao, Ruize Han +6

School of Computer Science, Northwestern Polytechnical University, Xi’an, China · Ningbo Institute, Northwestern Polytechnical University, Ningbo 315000, China · National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application Technology, Xi’an, China

Feature Fusion Multi-Object Tracking Multi-View Feature Fusion

Ask PDF

Abstract

Multi-Object Tracking

Explore similar work

Online Multi-Camera 3D Tracking via ID Prediction over Recurrent Sparse Queries

ModTrack: Sensor-Agnostic Multi-View Tracking via Identity-Informed PHD Filtering with Covariance Propagation

Beyond Appearance: A Multi-cue Framework and Large-scale Benchmark for Pedestrian Association and Tracking on Mobile Aerial-Ground Platforms