ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking

Haoyang Wu; Shoudong Han; Chaoyue Li; Sijia Chen; Zhenyang Xie; Sihan Wang

cs.CVSep 29, 2026

ReWorld-Track: A Recursive Event World Model for Language-Guided Multi-Camera Tracking

Authors: Haoyang Wu, Shoudong Han, Chaoyue Li, Sijia Chen, Zhenyang Xie, Sihan Wang

Organizations: Huazhong University of Science and Technology · Jiangxi University of Water Resources and Electric Power · Zhongnan University of Economics and Law

Abstract

Language-guided multi-camera tracking must preserve a target identity across unobserved gaps, where similar candidates and uncertain returns can make early associations unreliable. A wrong match can corrupt the history used to predict later observations and propagate identity errors across subsequent camera handoffs. We propose ReWorld-Track, a recursive event world model that carries association uncertainty into future predictions. Candidate matches and continued waiting define alternative target states, whose posterior probabilities are used to update a persistent recurrent belief. This representation preserves uncertainty about alternative trajectories through successive observations. This belief predicts the next camera, arrival time, and entry region, while appearance and language evidence guide association. By training across successive handoffs, the model learns to retain uncertainty that remains useful for later predictions and identity decisions. ReWorld-Track achieves HOTA scores of 65.19 on CityFlowV2 and 45.36 on MTMMC, with improved identity continuity across repeated handoffs. On MTMMC, its structured posterior update gains 0.50 HOTA points over a similarly sized generic updater and 0.94 points over fixed-moment soft association, raising next-camera accuracy from 86.03% to 87.41% and reducing median arrival-time error from 0.78 s to 0.71 s for subsequent target returns.

Figures & tables

Figure 1: Ambiguous identity continuation. (a) Last observation of GT375 in c017. (b) Camera hypotheses during the 8.8 s mask. (c) Target and competitor in the same c021 frame. White nodes denote masked streams. Blue dashed edges show route hypotheses, and coral markings show ground truth (GT). The panels follow the same target through this sequence.

Figure 2: Overview of ReWorld-Track. The scene connects the three steps to successive camera observations. Prediction guides candidate-or-null association. The resulting posterior updates the target belief. The green loop carries uncertainty into the next forecast. Bars and curves are illustrative.

CityFlowV2 : $D_{+}=3587$ , $D_{0}=14291$
Method	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	DetA $\uparrow$	AssA $\uparrow$	IDSW $\downarrow$	FM $\downarrow$
ReID	71.81	68.54	51.86	57.46	46.81	326	9.99
ReID + time	77.03	72.89	56.72	57.53	55.91	249	5.86
Reactive memory	80.23	76.62	61.48	57.68	65.53	191	6.72
TrackTA †	79.15	75.48	60.63	57.31	64.15	237	5.28
LaMMOn †	82.27	79.06	64.31	57.82	71.53	174	4.95

Table 1: Identity continuity with separate training by domain. Rates are percentages, and bold marks column optima. ReID denotes re-identification, BoT person-ReID and HC hierarchical clustering.

\dagger

marks our causal adaptation of an external tracker. Its results are re-evaluated with the causal initialization, test events and trajectory metrics specified in Section 4 .

Panel of Figure 3 — Figure 3: Missing-observation experiments. The GT375 timeline and six-camera graph illustrate masking. (a,b) Handoff accuracy versus mask duration and camera count. (c) Next-camera prediction by gap. (d) Arrival-time error CDF. No recur. omits feedback. Cam-time predicts camera and time.

Variant	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	FM $\downarrow$	Delay (s) $\downarrow$
ReWorld-Track	83.56 $\pm$ 0.23	58.12 $\pm$ 0.24	45.36 $\pm$ 0.20	4.90 $\pm$ 0.15	1.03 $\pm$ 0.02
w/o event prior	77.20 $\pm$ 0.23	52.61 $\pm$ 0.24	40.02 $\pm$ 0.19	7.41 $\pm$ 0.19	1.39 $\pm$ 0.03
w/o recursion	78.74 $\pm$ 0.25	54.36 $\pm$ 0.23	41.28 $\pm$ 0.19	6.81 $\pm$ 0.19	1.26 $\pm$ 0.03
w/o temporary null	85.05 $\pm$ 0.22	57.54 $\pm$ 0.20	44.98 $\pm$ 0.21	12.74 $\pm$ 0.25	0.61 $\pm$ 0.02
w/o query	81.72 $\pm$ 0.24	56.79 $\pm$ 0.21	44.10 $\pm$ 0.19	5.42 $\pm$ 0.17	1.09 $\pm$ 0.02

Table 2: MTMMC controls on 2749 handoffs and 10683 absence decisions. (a) Means and sample standard deviations over five seeds. (b) Three posterior updates under the same event predictor. Rates are percentages. Delay is in seconds. w/o means without.

CityFlowV2
Query condition	NQ	HG	F
Natural description	81.63	85.40	87.76
Partly unobservable	81.63	74.10	84.46
Misleading attributes	81.63	65.31	75.67
MTMMC
Natural description	78.39	80.16	84.28

Table 3: Controlled query perturbations on 637 vehicle and 509 pedestrian events. Values are HA (%). Bold marks each row’s best result for that condition.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Three query-initialized MTMMC s17 tracks with all sixteen streams available. The rows show GT20, GT97 and GT3 as $g_{1}$ , $g_{2}$ and $g_{3}$ . Columns show initialization, an intermediate observation, matching and continuation. The floor graph marks annotated transitions. Rose boxes show GT, mint dashed boxes predictions and blue boxes alternatives.

Figure 6: A vehicle instance of prediction, association and posterior feedback. The c023 and c025 observations condition the forecast, and the c026 candidates at 58.9 s supply new evidence. The posterior updates the persistent target state for the next handoff. Camera labels and times refer to this sequence. Figure 2 presents the corresponding general framework.

Figure 7: Language-conditioned vehicle association in CityFlowV2 S05. A query and source box initialize each persistent identity. The columns follow c026 before reappearance, after matching and during continuation. Rose GT ID labels identify targets 396/334/336, mint Pred ID labels show $g_{1}/g_{2}/g_{3}$ , and blue candidate letters identify alternatives within each image. Dots sample historical trails. Of $N=19$ cameras, $M=5$ are masked over nominal time 48.3–114.8 s inclusive, withholding RGB, detections, features and tracklet observations. The graph displays annotated visit order and co-visibility, highlighting the available c025/c026 views. For $g_{1}$ , the early null probabilities concern c026 while the target remains annotated and visible in the available c025 stream.

Figure 8: Paired pedestrian recovery under missing-camera observations on MTMMC s17. Identities, queries and source boxes match Figure 5 . Cameras c03, c10, c11 and c13 are masked over source frames 0–7353 inclusive. This withholds RGB, detections, appearance features and tracklet observations. Prior global state and physical routes remain. Of $N=16$ cameras, $M=4$ are masked, as indicated by the grey panels. The $g_{1}$ strip follows $P(\mathrm{null})$ among current candidates and temporary null, conditional on an active target, through the gap and reappearance. GT ID and Pred ID identify dataset targets and persistent display identities. Dots sample historical trails. Annotated transitions in the graph provide a reference for tracking under the installation graph.

	CityFlowV2			MTMMC
Variant	HA $\uparrow$	IDF1 $\uparrow$	FM $\downarrow$	HA $\uparrow$	IDF1 $\uparrow$	FM $\downarrow$
Full configuration	84.67	80.37	4.28	—	—	—
Reference configuration	—	—	—	82.10	56.43	5.34
w/o event prior	80.49	76.13	5.98	75.08	51.13	7.73
w/o arrival time	83.02	79.42	5.13	79.56	54.98	6.66
w/o entry region	83.94	80.58	4.54	78.87	54.36	6.58

Appendix

Table 4: Dataset-specific component ablations on the populations of Table 1 . CityFlowV2 uses the full configuration as its baseline. MTMMC uses the reference configuration. Rates are percentages. Table I adds HOTA, mean correct-acceptance delay,

\Delta

IDF1 and counts.

Variant	$N_{+}$	$N_{0}$	HOTA $\uparrow$	Delay (s)	$\Delta$ I (pp)
CityFlowV2
Full configuration	3037	612	65.19	1.16	0.00
w/o event prior	2887	854	60.59	1.27	-4.24
w/o arrival time	2978	733	64.12	1.61	-0.95
w/o entry region	3011	649	64.87	1.08	+0.21
w/o survival	2962	779	63.91	1.04	-1.20

Appendix

Table 5: Counts and trajectory metrics for the dataset-specific ablations in Table I . CityFlowV2 uses the full configuration, and MTMMC uses the reference configuration.

N_{+}

and

N_{0}

count correct handoffs and absence false accepts. Delay averages seconds over correct accepts.

\Delta

I is IDF1 minus the dataset’s unmodified baseline in percentage points.

Method	$D_{+}$	$N_{+}$	HA	$D_{0}$	$N_{0}$	FM
CityFlowV2
ReID	3587	2576	71.81	14291	1427	9.99
ReID + time	3587	2763	77.03	14291	838	5.86
Reactive memory	3587	2878	80.23	14291	961	6.72
TrackTA †	3587	2839	79.15	14291	754	5.28
LaMMOn †	3587	2951	82.27	14291	708	4.95

Appendix

Table 6: Counts for the principal baselines and dataset-specific ReWorld configurations. CityFlowV2 uses the full configuration, and MTMMC uses the reference configuration. Table P.2 includes the full MTMMC configuration. Correct handoffs

N_{+}

and false accepts

N_{0}

give

\mathrm{HA}=100N_{+}/D_{+}

and

\mathrm{FM}=100N_{0}/D_{0}

from the corresponding handoff and absence-decision populations.

	Correct count			HA (%)			$\Delta$ (pp)
Query variant	NQ	HG	F	NQ	HG	F	F–NQ
CityFlowV2
Natural description	520	544	559	81.63	85.40	87.76	+6.12
Paraphrase	520	536	552	81.63	84.14	86.66	+5.02
Weak attributes	520	506	529	81.63	79.43	83.05	+1.41
Partly unobservable	520	472	538	81.63	74.10	84.46	+2.83

Appendix

Table 7: Language contribution on paired query variants, using the same 637 CityFlowV2 and 509 MTMMC events across all five conditions. NQ omits language, HG uses a hard gate and F denotes the full model. Counts are correct handoffs and HA is in percent.

\Delta

is the full-minus-no-query difference in percentage points, calculated from correct-handoff counts before rounding.

Condition	Model	$D_{+}$	Correct	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	$\Delta$ HA
CityFlowV2
Matched scene	E	1173	974	83.03	79.68	64.72	0.00
	R	1173	986	84.06	80.69	65.52	+1.02
Unseen cameras	E	981	776	79.10	74.89	59.32	0.00
	R	981	791	80.63	76.58	60.31	+1.53
Unseen scene	E	1247	912	73.14	66.21	50.06	0.00

Appendix

Table 8: Within-domain transfer on condition-specific event groups for camera and scene shifts. E denotes LaMMOn on CityFlowV2 and CRTracker on MTMMC, while R denotes ReWorld-Track.

\Delta

HA compares R and E within the same condition. All models use target-domain training and validation-only selection before evaluation in each transfer condition.

Vehicles $\to$ Pedestrians
Method	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	FM $\downarrow$
Appearance + time	70.07	47.26	36.18	7.29
ReWorld-Track	74.68	50.34	39.27	5.14
Target-trained	84.61	58.43	46.39	3.99
Pedestrians $\to$ Vehicles
Appearance + time	72.09	70.18	56.24	6.63

Appendix

Table 9: Cross-domain transfer. Rates are percentages. Target-trained rows use target labels. Other rows use source training and selection.

Method	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	IDSW $\downarrow$	FM $\downarrow$
Vehicles : $D_{+}=3200$ , $D_{0}=12800$
Strong baseline A	83.28	79.18	64.13	173	5.13
Strong baseline B	83.97	80.06	65.21	162	4.59
ReWorld-Track	85.28	81.24	66.37	149	3.93
Pedestrians : $D_{+}=2800$ , $D_{0}=11200$
Strong baseline A	82.29	56.32	44.16	334	5.62

Appendix

Table 10: Additional domain-level comparison on the event groups in Appendix N . Baseline A uses reactive appearance/time memory and B uses one-way event prediction. Rates are percentages, with counts in Table N . Bold marks the best result for each metric within the corresponding domain.

Method	$D_{+}$	$N_{+}$	HA	$D_{0}$	$N_{0}$	FM
Vehicles
Strong baseline A	3200	2665	83.28	12800	657	5.13
Strong baseline B	3200	2687	83.97	12800	588	4.59
ReWorld-Track	3200	2729	85.28	12800	503	3.93
Pedestrians
Strong baseline A	2800	2304	82.29	11200	629	5.62

Appendix

Table 11: Counts for Table N .

N_{+}

counts correct handoffs and

N_{0}

counts absence false accepts. The rates are

\mathrm{HA}=100N_{+}/D_{+}

and

\mathrm{FM}=100N_{0}/D_{0}

, rounded to two decimals.

Method	ADE $\downarrow$	FDE $\downarrow$	Occ. IoU $\uparrow$	Chamfer $\downarrow$
Scene baseline A	1.23	2.57	51.86	0.27
Scene baseline B	0.99	2.13	58.72	0.19
ReWorld-Track	0.83	1.78	64.35	0.16

Appendix

Table 12: Auxiliary pedestrian spatial evaluation with a 3 s horizon, six forecast samples and 0.20 m occupancy-grid cells. Average displacement error (ADE), final displacement error (FDE) and Chamfer distance use metres. Occupancy intersection over union (Occ. IoU) uses percent. Baselines A and B are constant-velocity and GRU predictors with the same geometry supervision.

Outcome	Count
Both methods correct	2200
ReWorld-Track correct, CRTracker incorrect	57
ReWorld-Track incorrect, CRTracker correct	43
Both methods incorrect	449

Appendix

Table 13: Paired outcomes for ReWorld-Track and CRTracker on 2749 events for the MTMMC reference configuration, separating shared outcomes from events on which the methods disagree.

Comparison	Observed difference	95% interval / calculation
MTMMC HA comparison	14 / 2749 (+0.51 pp)	Paired event-level Wald 95% interval -0.20 to +1.22 pp. The exact McNemar test gives p = 0.1933 from paired counts in Table P.1 .
MTMMC IDF1 comparison	+0.52 pp	95% interval for +0.52 pp is [-0.18, +1.22]. Resampling uses the trajectory-level identity aggregation .
MTMMC HOTA comparison	-0.34 pp	95% interval for -0.34 pp is [-1.01, +0.33]. The methods share the HOTA evaluation and resampling protocol .
Mechanism FM comparison	-2.07 pp	95% interval for -2.07 pp is [-2.71, -1.43]. Calculation uses the paired absence-decision records .
Mechanism delay comparison	+0.35 s	95% interval for +0.35 s is [+0.18, +0.52]. Correct-acceptance sample counts are reported for each method. Common-correct events are analyzed separately .

Appendix

Table 14: Uncertainty for reference comparisons. MTMMC rows compare the reference configuration with CRTracker. Mechanism rows compare the full model with recursive threshold. HA uses the displayed event-level calculation, while other intervals use paired trajectory or decision analyses. Table P.4 evaluates training-run variation separately.

Variant	E	FB	N	HA $\uparrow$	HOTA $\uparrow$	FM $\downarrow$	Delay (s) $\downarrow$
Memory + spatiotemporal prior	–	–	–	81.21	43.17	6.53	1.13
One-way event + threshold	Y	–	–	82.18	44.28	5.58	1.21
Recursive event + threshold	Y	Y	–	85.04	45.71	6.06	0.94
ReWorld-Track	Y	Y	Y	84.61	46.39	3.99	1.29

Appendix

Table 15: Standalone MTMMC RGB mechanism study on 2800 handoffs and 11200 absence decisions. E, FB and N denote event prediction, posterior feedback and explicit null. Y and – indicate enabled and disabled components. Rates are percentages and delay is mean correct-acceptance time in seconds. Bold marks column optima across the four variants.

Method / evaluation	N+	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	IDSW $\downarrow$	N0	FM $\downarrow$
CRTracker	2243	81.59	55.91	44.12	339	606	5.67
Reference configuration	2257	82.10	56.43	43.78	324	571	5.34
Full configuration	2297	83.56	58.12	45.36	298	524	4.90

Appendix

Table 16: ReWorld reference and full configurations and CRTracker on the same fixed MTMMC manifest of

D_{+}=2749

handoffs and

D_{0}=10683

absence decisions. Event IDs and decision timestamps are identical. Rates are percentages, and Table P.4 reports five-seed HOTA variation.

Variant	N+	HA $\uparrow$	IDF1 $\uparrow$	HOTA $\uparrow$	N0	FM $\downarrow$
One-way events + threshold	2208	80.32	56.05	43.62	665	6.22
Standard soft association	2261	82.25	57.06	44.42	591	5.53
Learned feedback + threshold	2325	84.58	57.81	44.91	711	6.66
Learned feedback + null	2297	83.56	58.12	45.36	524	4.90

Appendix

Table 17: Predictor-matched MTMMC controls on

D_{+}=2749

handoffs and

D_{0}=10683

absence decisions. Fixed-moment soft association and learned feedback use the same event predictor.

Seed / summary	CRTracker	ReWorld-Track	Paired difference
11	43.96	45.10	+1.14
23	44.21	45.42	+1.21
37	44.08	45.21	+1.13
51	44.16	45.55	+1.39
71	44.19	45.52	+1.33
Mean $\pm$ sample SD	44.12 $\pm$ 0.10	45.36 $\pm$ 0.20	1.24 $\pm$ 0.12

Appendix

Table 18: Five matched training seeds for CRTracker and ReWorld-Track. Each row pairs the same seed across methods. All runs share identical handoff event IDs and absence-decision timestamps. Mean and sample SD describe training-run variation on this fixed test manifest.

Threshold	N+	HA $\uparrow$	N0	FM $\downarrow$	HOTA $\uparrow$	Delay (s)
0.50	2340	85.12	1004	9.40	44.69	0.72
0.65	2325	84.58	760	7.11	45.07	0.86
0.80	2297	83.56	524	4.90	45.36	1.03
0.90	2231	81.16	384	3.59	45.12	1.24
0.95	2115	76.94	266	2.49	44.28	1.52

Appendix

Table 19: Updated-model commitment thresholds on fixed populations

D_{+}=2749

and

D_{0}=10683

. Delay averages seconds over

N_{+}

correct accepts. The sweep describes test behavior at fixed thresholds. Operating-point selection uses validation data.

Gap (s)	Events	Full N+	Full HA	Memory N+	Memory HA	$\Delta$ HA (pp)
1	600	572	95.33	557	92.83	2.50
2	600	565	94.17	535	89.17	5.00
4	600	552	92.00	494	82.33	9.67
6	600	540	90.00	451	75.17	14.83
8.8	600	527	87.83	403	67.17	20.67

Appendix

Table 20: Missing-camera duration study with 600 fixed events per condition and identical event masks across methods. Event membership stays fixed as mask duration increases. HA is in percent.

Natural blind gap	Reactive memory	One-way event	ReWorld-Track	Full - Memory
0.5–2 s	86.9	87.8	89.4	+2.5
2–4 s	80.2	82.7	86.0	+5.8
4–8 s	71.1	74.9	80.8	+9.7
>8 s	57.4	63.8	72.9	+15.5

Appendix

Table 21: MTMMC HA (%) by natural blind-gap duration with cameras available. Bins are fixed independently of outcomes. Methods share the evaluation protocol and are compared within each duration bin using the events assigned to that interval of natural non-observation.

Subset	No query	Hard gate	Natural query	Paraphrase	Misleading
Easy appearance	90.1	90.8	92.2	91.7	84.9
Appearance-ambiguous	63.2	67.4	74.1	72.8	57.0

Appendix

Table 22: MTMMC HA (%) under appearance ambiguity. A fixed margin between the correct candidate and strongest appearance distractor defines the subsets. Query variants use the same events within each subset to measure the effect of language under fixed visual ambiguity.

Method	NLL $\downarrow$	Brier $\downarrow$	ECE $\downarrow$
Raw similarity score	0.512	0.181	8.7%
Temperature-scaled scorer	0.447	0.156	5.0%
ReWorld joint belief	0.391	0.127	2.8%

Appendix

Table 23: Calibration of candidate/null association belief for the full MTMMC configuration. Negative log-likelihood (NLL), Brier squared-probability error and ECE measure probability quality. Lower values indicate better calibrated candidate/null probabilities.

Setting	Cameras	Active IDs	Candidates / update	Gated pairs / update
Light	6	8	12	46
Medium	12	32	38	196
Heavy	19	64	73	412
Full 19-camera	19	48	71	1186

Appendix

Table 24: Runtime on one RTX 4090 (24 GB), batch size 1 and FP16, with 100 warm-up and 1000 measured updates. Core timing starts from cached detections, causal prefixes and CLIP features. End-to-end (E2E) timing includes RGB, detection, local tracking, CLIP and the core. P95 denotes 95th-percentile latency, and memory reports the core workload. Candidates count all camera-local candidates. Gated pairs count identity–candidate hypotheses retained after spatiotemporal gating. Core cost depends on these pairs, graph propagation and posterior updates.

Setting	Reactive memory	One-way event	ReWorld-Track
Seen topology	82.3	83.4	85.1
Unseen cameras	76.8	78.3	80.6
Unseen topology	67.9	70.8	74.6

Appendix

Table 25: MTMMC HA (%) under camera and installation-topology shift. Settings share the evaluation protocol while changing camera and topology exposure within MTMMC.

Step	Operation
1	Receive current camera availability and update causal local-track prefixes. Form the candidate set with the appearance and motion evidence received so far.
2	Propagate the previous posterior over elapsed time with the fixed camera graph and transition kernel. This predicts the state before receiving current evidence.
3	Integrate the preceding observation-event forecast over the newly elapsed interval to obtain camera-local candidate and null prior masses.
4	Compute target-versus-background evidence for current candidates and normalize jointly with temporary null to assign probability mass to every association alternative.
5	Apply the association rule and per-camera exclusivity constraints. Assign each local observation at most once across the active target identities.
6	Update the active-state mixture and network-presence probability, preserve the global identity, store the posterior and form the next event forecast.

Appendix

Table 26: Online update procedure for one target. All steps use observations and availability up to the current decision time to advance one persistent identity through the prediction and association cycle.

Update	Input	Output dim.	Params (M)	Latency (ms)
Fixed-moment soft	Same	256	0	0.11
Generic learned	Same	256	0.64	0.39
ReWorld update	Same	256	0.66	0.41

Appendix

Table 27: Posterior-update module size and cost. All updates receive the same information. Parameter counts exclude shared modules. Latency is measured on an RTX 4090 at batch size 1 in FP16, after 100 warm-up iterations and over 1000 measured updates.

Test scene	Fixed soft	Generic learned	ReWorld	$\Delta$
Scene A	44.42 $\pm$ 0.09	44.86 $\pm$ 0.08	45.36 $\pm$ 0.20	+0.50
Scene B	42.98 $\pm$ 0.12	43.39 $\pm$ 0.13	43.86 $\pm$ 0.19	+0.47
Scene C	46.21 $\pm$ 0.11	46.66 $\pm$ 0.14	47.12 $\pm$ 0.22	+0.46
Scene mean	44.54	44.97	45.45	+0.48

Appendix

Table 28: HOTA in three scene-held-out evaluations. Entries are means

\pm

sample SDs over five independent training seeds. The last row averages the scene means.

\Delta

compares ReWorld with the generic learned updater within each held-out scene and in the average across scenes.

Update	Camera Top-1 $\uparrow$	Arrival MedAE $\downarrow$	Arrival P90 $\downarrow$
Fixed-moment soft	84.62	0.86	2.41
Generic learned	86.03	0.78	2.22
ReWorld update	87.41	0.71	2.07

Appendix

Table 29: Prediction following the posterior update. Next-camera Top-1 is a percentage. MedAE and P90 are the median and 90th percentile of absolute arrival-time error, in seconds.

Update	IR@1 $\uparrow$	IR@2 $\uparrow$	IR@3 $\uparrow$
No recursion / reset	95.84	89.63	82.24
Fixed-moment soft	98.21	95.18	90.87
Generic learned	98.46	96.39	93.14
ReWorld update	98.73	97.42	95.79

Appendix

Table 30: Identity retention IR@

k

(%) after

k

successive handoffs.

Explore similar work

Jul 19, 2026cs.CV

VLA-ReID: Video-Level Association for Re-Identification in Multi-Object Tracking with Highly Similar Objects

Multi-object tracking (MOT) aims to localize multiple objects in videos while preserving their identities over time. Long-term identity preservation remains difficult when objects are small, densely distributed, and highly similar in appearance, as in bee swarm scenes. Existing trackers rely on re-identification (re-ID) models trained through single-instance assignment (instance-level querying). At inference, however, MOT requires global assignment between multiple trajectories and detections, corresponding to video-level querying. This training-inference mismatch can cause identity switches among visually similar objects. Existing approaches also often require substantial additional annotations to enhance appearance discrimination. We propose Video-Level Association re-ID (VLA-ReID), which reformulates re-ID as video-level association modeling. It uses aggregated historical trajectory features as queries and all current-frame detections as candidates, enabling direct optimization of their global association at each frame. In addition, Frame-Common Appearance Estimation (FCAE) estimates a common appearance direction from current-frame detections, while Common-Appearance Suppression (CAS) removes the corresponding component along this direction from trajectory and detection features. This amplifies discriminative differences among highly similar objects without additional annotations. Experiments on BEE24 show that VLA-ReID improves HOTA by 1.1, MOTA by 0.3, AssR by 2.6, AssA by 0.7, and IDF1 by 0.8 over state-of-the-art trackers, while reducing identity switches by 28%. These results demonstrate the effectiveness of video-level re-ID modeling for appearance-based association in MOT.

Yanrong Qin, Xiaoyan Cao, Yao Yao

Glasgow College, University of Electronic Science and Technology of China, Chengdu, China · Key Laboratory for Urban Habitat Environmental Science and Technology, School of Environment and Energy, Peking University Shenzhen Graduate School, Shenzhen, China · School of Management Science and Real Estate, Chongqing University, Chongqing, China

Multi-Object Tracking Re-Identification Fine-Grained Video Understanding Object Detection Associations Swarms

Ask PDF

Sep 1, 2026cs.CV

Beyond the Image Plane: World-Grounded Queries for Multi-Object Tracking

Monocular videos record 3D scenes as sequences of 2D image-plane projections, obscuring depth and spatial relationships. Multi-object trackers localize and associate objects primarily using appearance and geometry observed only in the image plane, inheriting these ambiguities. To address this limitation, we introduce PLANET, an end-to-end multi-object tracker designed to move beyond the image plane. As an enabling step, we lift existing 2D tracking datasets into 3D. We then form world-grounded queries by embedding reconstructed 3D scene geometry into the features and positional encodings used during query formation. An auxiliary 3D location prediction task further encourages the queries to encode object positions during training. A complementary dual-resolution temporal memory preserves this evidence across longer temporal gaps. As a result, PLANET achieves state-of-the-art performance across three diverse benchmarks.

Orcun Cetintas, Guillem Brasó, Tim Meinhardt +1

1NVIDIA · 2Technical University of Munich

Multi-Object Tracking Monocular Video Planar Queries

Ask PDF

Sep 5, 2026cs.CV

CST-WM: A Causally Structured World Model for Embodied Visual Tracking

Embodied visual tracking requires a robot to choose actions that keep a moving target observable at a suitable distance, and to recover it after occlusion, out-of-view drift, or distractor crossings. We cast the task as planning over future target evidence with an action-conditioned world model. In logged tracking data, however, the behavior policy's actions are correlated with where the target is, so a generic predictor can learn a shortcut: it writes the current action directly into its prediction of target evidence, instead of letting the action affect that evidence only by moving the robot and changing what it observes. We call this failure causal hallucination; the resulting rollouts look plausible but rank candidate actions for the wrong reason. We propose CST-WM, a causally structured world model whose state is split into target-evidence, robot, and observation branches. Its transition removes the same-step edge from action to target evidence but keeps the path through robot motion and the resulting views, so candidate actions are still distinguished by their predicted ego-motion. With rollout-based model-predictive control, a single model handles both steady following and re-acquisition after target loss. On EVT-Bench and Habitat 3.0, covering standard tracking, target-loss recovery, and cross-dataset transfer, CST-WM improves following, distance-range control, safety, and re-acquisition over reactive trackers and world-model baselines, and removing the action mask causes the largest drop in re-acquisition among our ablations. Offline, CST-WM has lower multi-step rollout error, and its ranking of candidate actions agrees better with the simulator's. On a Unitree Go2 quadruped, CST-WM succeeds in 20 of 30 real-world trials under occlusion, distractor crossing, and fast motion, against 14 for TrackVLA.

Junyi Hu, Shuaihang Yuan, Jiazhao Liang +1

New York University Abu Dhabi

Efficient World-Action Model World Models Multi-Object Tracking Model Predictive Control Distractor

Ask PDF