AR2-4FV: Anchored Referring and Re-identification for Long-Term Grounding in Fixed-View Videos

Teng Yan; Yihan Liu; Jiongxu Chen; Teng Wang; Jiaqi Li; Bingzhuo Zhong

cs.CVMar 8, 2026

AR2-4FV: Anchored Referring and Re-identification for Long-Term Grounding in Fixed-View Videos

Authors: Teng Yan, Yihan Liu, Jiongxu Chen, Teng Wang, Jiaqi Li, Bingzhuo Zhong

Organizations: The Hong Kong University of Science and Technology (Guangzhou)

Abstract

Long-term language-guided referring in fixed-view videos is challenging: the referent may be occluded or leave the scene for long intervals and later re-enter, while framewise referring pipelines drift as re-identification (ReID) becomes unreliable. AR2-4FV leverages background stability for long-term referring. An offline Anchor Bank is distilled from static background structures; at inference, the text query is aligned with this bank to produce an Anchor Map that serves as persistent semantic memory when the referent is absent. An anchor-based re-entry prior accelerates re-capture upon return, and a lightweight ReID-Gating mechanism maintains identity continuity using displacement cues in the anchor frame. The system predicts per-frame bounding boxes without assuming the target is visible in the first frame or explicitly modeling appearance variations. AR2-4FV achieves +10.3% Re-Capture Rate (RCR) improvement and -24.2% Re-Capture Latency (RCL) reduction over the best baseline, and ablation studies further confirm the benefits of the Anchor Map, re-entry prior, and ReID-Gating.

Figures & tables

Panel of Figure 2 — Figure 2: Dataset statistics: video duration distribution, target re-entry frequency, and scene category composition.

Figure 3: Diverse language-guided referring queries and annotated referents in AR 2 -4FV-Bench. Examples span indoor and outdoor fixed-view scenes as well as cinematic clips. Queries cover anchor-referential descriptions (e.g., “the person near the doorway”) and attribute-based disambiguation by color, clothing, or pose. Green boxes indicate ground-truth referents.

Figure 4: Overview of AR 2 -4FV. Offline: static structures are distilled into an Anchor Bank. Online: the query is aligned to the bank to produce a persistent Anchor Map; this map generates proposals and drives the search and re-entry prior $P^{\mathrm{re}}$ . With mask-aware pooling we compute a fusion score, and ReID-Gating validates candidates using appearance similarity, anchor evidence, and displacement in the anchor frame, yielding per-frame boxes $\{y_{t}\}$ .

Model	Reference	IDF1 $\uparrow$	RCR $\uparrow$	RCL $\downarrow$
MTTR	[CVPR’22]	56.3	0.60	33.8
ReferFormer	[CVPR’22]	57.9	0.63	31.2
OnlineRefer	[ICCV’23]	58.6	0.64	29.9
JointNLT	[CVPR’23]	59.4	0.63	30.8
SOC	[NIPS’23]	58.7	0.64	30.3
VideoLISA	[NIPS’24]	57.2	0.60	29.5

Table 1: Quantitative comparison of AR 2 -4FV and state-of-the-art models on the AR 2 -4FV-Bench.

Model	Backbone	Precision $\uparrow$					mAP $\uparrow$	mIoU $\uparrow$
Model	Backbone	[email protected]	[email protected]	[email protected]	[email protected]	[email protected]	mAP $\uparrow$	mIoU $\uparrow$
MTTR	Video-Swin-T	72.1	68.0	60.3	45.6	1.2	44.6	64.1
ReferFormer	Video-Swin-T	70.5	66.9	58.8	42.7	0.9	42.8	62.9
OnlineRefer	Video-Swin-B	75.4	71.4	63.6	48.5	0.8	46.1	64.2
JointNLT	ResNet-50	74.2	69.8	59.5	42.1	0.8	43.5	61.9
SOC	Video-Swin-T	69.2	65.8	57.2	41.5	0.7	41.7	61.6

Table 2: Comparison of Precision between AR 2 -4FV and state-of-the-art models on the AR 2 -4FV-Bench.

Components				mIoU $\uparrow$	mAP $\uparrow$	IDF1 $\uparrow$	RCR $\uparrow$	RCL $\downarrow$
Baseline	Anchor Map	Re-ID Gating	Re-entry Prior	mIoU $\uparrow$	mAP $\uparrow$	IDF1 $\uparrow$	RCR $\uparrow$	RCL $\downarrow$
✓				63.2	45.2	61.2	0.67	27.1
✓	✓			-	-	-	-	-
✓	✓	✓		64.7	46.3	62.2	0.70	26.9
✓	✓		✓	63.8	45.5	61.3	0.68	21.3
✓	✓	✓	✓	66.9	49.2	64.8	0.75	20.1

Table 3: Ablation study of AR 2 -4FV with component-wise configurations.

Query Re-Identification

See all 9 papers

SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs
Mar 12, 2026Mohamad Alansari, Naufal Suryanto, Divya Velayudhan +3Multimodal Large Language Models Vision-Language Grounding
Khalifa University, Abu Dhabi, UAE
Event-Driven Refresh and Recurrence Memory to Reduce Stale Grounding in Referring Video Object Segmentation
Sep 30, 2026Abu Hanif Muhammad Syarubany, Jaehyun Jang, Siwoo Lim +2Video Object Segmentation Referring Video Object Segmentation
School of Electrical Engineering, Korea Advanced Institute of Science & Technology (KAIST), Daejeon 34141, Republic of Korea
RefCaptioner: Multi-Reference Image-Grounded Video Captioning
Jul 30, 2026Tengfei Liu, Yang Shi, Yuran Wang +16Video Captioning Vision-Language Grounding
PKU · Kling Team · NUS +7

Mar 12, 2026J/K move · Enter open · S save

SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs

Mohamad Alansari, Naufal Suryanto, Divya Velayudhan, Sajid Javed+2

Khalifa University, Abu Dhabi, UAE

Multimodal Large Language Models Vision-Language Grounding Referring Video Object Segmentation Temporal Video Grounding

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have advanced from image-level reasoning to pixel-level grounding, but extending these capabilities to videos remains challenging as models must achieve spatial precision and temporally consistent reference tracking. Existing video MLLMs often rely on a static segmentation token ([SEG]) for frame-wise grounding, which provides semantics but lacks temporal context, causing spatial drift, identity switches, and unstable initialization when objects move or reappear. We introduce SPARROW, a pixel-grounded video MLLM that unifies spatial accuracy and temporal stability through two key components: (i) Target-Specific Tracked Features (TSF), which inject temporally aligned referent cues during training, and (ii) a dual-prompt design that decodes box ([BOX]) and segmentation ([SEG]) tokens to fuse geometric priors with semantic grounding. SPARROW is supported by a curated referential video dataset of 30,646 videos and 45,231 Q&A pairs and operates end-to-end without external detectors via a class-agnostic SAM2-based proposer. Integrated into three recent open-source video MLLMs (UniPixel, GLUS, and VideoGLaMM), SPARROW delivers consistent gains across six benchmarks, improving up to +8.9 J&F on RVOS, +5 mIoU on visual grounding, and +5.4 CLAIR on GCG. These results demonstrate that SPARROW substantially improves referential stability, spatial precision, and temporal coherence in pixel-grounded video understanding. Project page: https://risys-lab.github.io/SPARROW

Abstract

Query Re-Identification

Explore similar work

SPARROW: Learning Spatial Precision and Temporal Referential Consistency in Pixel-Grounded Video MLLMs

Event-Driven Refresh and Recurrence Memory to Reduce Stale Grounding in Referring Video Object Segmentation

RefCaptioner: Multi-Reference Image-Grounded Video Captioning