WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding

Chenxu Wang; Hangyi Kuang; Bo-Wen Yin; Yuxuan Li; Quan-Sheng Zeng; Yang Zhang; Xu Guo; Xiang Li

cs.CVOct 14, 2026

WinARDet: Compressed Window Auto-Regression for Long-Horizon Vision-Language Grounding

Authors: Chenxu Wang, Hangyi Kuang, Bo-Wen Yin, Yuxuan Li, Quan-Sheng Zeng, Yang Zhang, Xu Guo, Xiang Li,

Organizations: Nankai University · Beijing Normal University

Abstract

MLLM-based detectors formulate vision-language grounding as autoregressive coordinate generation, yet in dense scenes, they often collapse into a box repetition loop, repeatedly emitting identical boxes until the decoding budget is exhausted. We trace this failure to historical-answer attention takeover: Under full-history autoregression, repeated coordinate tokens in the growing answer history attract disproportionate attention, crowding out the attention sinks needed to attend to new visual evidence. % To break this loop, in this paper, we propose Compressed Window Auto-Regression (WinAR). This new mechanism partitions a long answer sequence into consecutive generation windows and restricts full token-level autoregression to the current window only while compressing each previous window into a fixed set of latent tokens that condition subsequent decoding. This design reduces the repeated exposure to raw historical coordinates that triggers the attention takeover, yet preserves cross-window dependencies essential for coherent multi-object generation. On RSDense, which contains 200-2,000 objects per image, our model WinARDet improves RexOmni F1-score (IoU=0.5) from 32.16 to 39.36 over the baseline, while reducing the redundancy rate from 35.48% to 13.00%. WinARDet also generalizes across decoding strategies and output formats, including parallel box decoding and polygon generation, providing a practical paradigm for long-horizon vision-language grounding across scene densities.

Figures & tables

Figure 1: WinARDet overview. Grounding Performance across Densities(Left): WinARDet outperforms the baseline across benchmarks of varying scene densities( Lin et al. (2014) ; Xia et al. (2018) ; Cheng et al. (2023) ; Goldman et al. (2019) ), with increasingly pronounced gains in denser scenes. Breaking Box Repetition Loop(Middle): WinARDet breaks the box repetition loop, suppressing repeated predictions and recovering substantially more distinct objects. Beyond Bounding Boxes(Right): WinARDet generalizes to polygon detection, extending beyond bounding-box.

Figure 2: (a) Metrics are averaged over intervals of 100 predicted boxes on RSDense(Section 4 ). FHAR(left) rapidly accumulates redundant predictions as new-object discovery collapses, whereas WinAR(right) suppresses redundancy and sustains the discovery of new objects. (b) The onset of the box repetition loop coincides with historical-answer attention takeover and reduced system-token attention in FHAR(left), while WinAR(right) maintains a more stable attention allocation.

Figure 3: Illustration of the serialized response formats and compression granularities for RexOmni and LocateAnything. RexOmni represents the answer as a flat token sequence and generates it through next-token prediction, whereas LocateAnything organizes the answer into box-aligned blocks and predicts the coordinate tokens within each block in parallel. Accordingly, WinAR partitions RexOmni outputs into fixed-length token windows (e.g., every 300 tokens), while forming windows from a fixed number of complete blocks (e.g., every 50 blocks) for LocateAnything.

Figure 4: Overview of the proposed Compressed Window Auto-Regression. (a) A long answer is partitioned into consecutive windows and processed sequentially. Preceding windows are compressed into memory and combined with the multimodal prefix to condition the current window, on which cross-entropy loss is applied. (b) Structure of the Window History Compressor.

Method	Mode	DOTA2 & SODA-A					RSDense
Method		R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$
RexOmni ( Jiang et al., 2026 )
Baseline		NTP	54.90	62.56	58.48	–	–	17.32	21.68	12.69	–	–
FHAR	NTP	63.86	61.66	62.74	4.58%	3.24	23.39	24.99	16.57	58.74%	5.85
FHAR*	NTP	66.94	68.26	67.59	2.43%	1.39	36.82	37.92	32.16	35.48%	3.14
FHAR*-GRPO	NTP	66.61	67.77	67.18	2.31%	1.71	37.27	37.24	31.98	35.05%	3.11

Table 1: Comparison on DOTA2 & SODA-A and the extremely dense RSDense subset.

\mathrm{R}

\mathrm{P}

, and

\mathrm{F1}

denote recall, precision, and F1 score at an IoU threshold of 0.5. Mean

R_{\mathrm{dup}}

and Mean

R_{\mathrm{over}}

measure duplicate predictions and over-prediction, respectively. The best and second-best results within each comparison group are highlighted in bold and underline , respectively. NTP denotes next token prediction, PBD denotes parallel box decoding, and Hybrid denotes a mixed decoding mode combining NTP and PBD. See Locate Anything ( Wang et al., 2026b ) for details.

RexOmni	box / s $\uparrow$	valid box / s $\uparrow$	Training Peak GPU Memory (GB) $\downarrow$	Ship			Small Vehicle
RexOmni	box / s $\uparrow$	valid box / s $\uparrow$	Training Peak GPU Memory (GB) $\downarrow$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5
FHAR*	6.05	3.57	57.62	57.13	47.23	51.71	22.65	17.80	19.94
WinAR	6.68	5.40	42.97	65.35	58.50	61.74	30.06	24.45	26.96

Table 2: Comparison of inference speed on RSDense and performance on two extremely dense categories. Boxes/s denotes the average number of predicted boxes per image per second, reflecting the decoding throughput. Valid Boxes/s measures the throughput after removing redundant boxes. Inference speed is measured on eight NVIDIA H100 GPUs.

Method	DOTA2					SODA-A
Method	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$
Baseline	2.49	3.06	2.44	–	–	1.95	2.27	2.03	–	–
FHAR	46.79	41.82	39.67	8.93%	8.03	47.48	27.19	32.78	7.16%	6.32
FHAR*	51.20	47.98	47.25	6.34%	4.99	54.45	34.71	38.64	3.72%	3.27
WinAR	52.04	49.30	48.93	2.46%	0.89	55.35	39.38	44.00	1.93%	0.75

Table 3: Polygon detection results on DOTA2 and SODA-A. The Baseline is RexOmni model.

WinAR			DOTA2 & SODA-A					RSDense
$L$	$B$	$N$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$
300	6	8	68.67	68.41	68.49	1.21%	0.48	48.59	42.82	44.78	5.38%	0.66
200	4	6	67.39	67.61	67.22	0.65%	0.42	47.93	42.60	44.56	4.21%	0.57
200	0	20	63.40	67.16	64.82	0.28%	0.18	27.96	39.59	32.47	2.59%	0.04
100	2	4	68.40	66.15	67.04	0.12%	0.27	47.04	41.41	43.71	0.89%	0.32

Table 4: Ablation study of WinAR with different window length

L

, boundary size

B

, and number of compressed memory tokens

N

Compressor Layers	DOTA2 & SODA-A					RSDense
Compressor Layers	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$	R@IoU $\uparrow$ 0.5	P@IoU $\uparrow$ 0.5	F1@IoU $\uparrow$ 0.5	Mean $R_{\mathrm{dup}}\downarrow$	Mean $R_{\mathrm{over}}\downarrow$
8	66.17	63.14	64.62	1.31%	0.66	45.75	38.23	39.32	16.87%	1.28
6	68.32	67.63	67.97	1.26%	0.51	45.23	38.90	39.36	13.00%	0.87
4	66.57	64.45	65.49	1.23%	0.44	43.60	38.20	37.73	15.64%	0.78

Table 5: Ablation study on the number of layers in the Window History Compressor.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	#Images	#Annotations	Min / Image	Max / Image	Avg. / Image
DOTA2 & SODA-A	45,628	1,493,991	1	2,831	32.74
DOTA2-Poly	13,447	473,548	1	2,831	35.22
SODA-A-Poly	32,181	1,020,443	1	2,653	31.71
SKU110K	8,769	1,293,724	1	718	147.53
COCO	117,266	860,001	1	93	7.33

Appendix

Table 6: Detailed statistics of the training datasets used in our experiments.

Dataset	#Images	#Annotations	Min Objects/Image	Max Objects/Image	Avg. Objects/Image
DOTA2 & SODA-A	18,707	656,706	1	2,305	35.14
RSDense	712	293,371	201	2,305	412.04
SKU110K	2,920	429,411	20	533	147.06
DOTA2-Poly	4,053	140,103	1	2,307	34.57
SODA-A-Poly	14,654	517,099	1	1,692	35.29
COCO	4,952	36,781	1	63	7.43

Appendix

Table 7: Detailed statistics of the test datasets used in our experiments.

Hyperparameter	RO-FHAR	RO-FHAR*	RO-WinAR	LA-FHAR*	LA-WinAR
Training Epochs	4	4	4	4	4
Oversample	False	True	True	True	True
Optimize Steps	1,424	1,864	1,864	2,004	2,004
Per-device Batch Size	1	1	1	1	1
Gradient Accumulation Steps	16	16	16	16	16
Base Learning Rate	$4\times 10^{-5}$	$4\times 10^{-5}$	$4\times 10^{-5}$	$4\times 10^{-5}$	$4\times 10^{-5}$

Appendix

Table 8: Training hyperparameters for RexOmni and LocateAnything. RO and LA denote RexOmni and LocateAnything, respectively.

Dataset	Method	[email protected] $\uparrow$	[email protected] $\uparrow$	[email protected] $\uparrow$
COCO	FHAR*	63.28	67.84	65.48
	WinAR	64.31 $(+1.03)$	67.87 $(+0.03)$	66.04 $(+0.56)$
DOTA2 & SODA-A	FHAR*	62.08	66.94	64.42
	WinAR	62.79 $(+0.71)$	68.40 $(+1.46)$	65.17 $(+0.75)$
SKU110K	FHAR*	53.89	81.75	64.96
	WinAR	54.93 $(+1.04)$	80.94 $(-0.81)$	65.44 $(+0.48)$

Appendix

Table 9: Performance comparison between RexOmni-FHAR ∗ and RexOmni-WinAR on COCO, DOTA2 & SODA-A, RSDense, and SKU110K under the all-prediction evaluation setting. RexOmni-FHAR ∗ denotes the RexOmni-FHAR baseline trained with long-sample oversampling.

Method	Mode	Boxes/s $\uparrow$	Valid Boxes/s $\uparrow$	Training Peak GPU Memory (GB) $\downarrow$
RexOmni–FHAR ∗	NTP	6.05	3.57	57.62
RexOmni-WinAR	NTP	6.68	5.40	42.97
Locate Anything–FHAR ∗	NTP	5.46	5.14	67.38
Locate Anything-WinAR	NTP	5.40	5.37	44.43
Locate Anything-FHAR ∗	PBD	29.03	24.63	67.38
Locate Anything-WinAR	PBD	26.15	25.61	44.43

Appendix

Table 10: Comparison of inference throughput and training peak GPU memory. Boxes/s denotes the average number of predicted boxes per image per second, while Valid Boxes/s measures the throughput after removing redundant predictions. Inference speed is measured on eight NVIDIA H100 GPUs.

Figure 5: Visualization Comparison in Dense Scenes.

Figure 6: Visualization Comparison in Dense Scenes.

Figure 7: Visualization Comparison in Polygon Detection.

Figure 8: Visualization Comparison in Polygon Detection.

Vision-Language Grounding

Vision-Language Grounding links words and phrases to the objects, regions, actions, or moments they describe in images, video, and scenes.

See all 418 papers

GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
Sep 30, 2026Qize Yu, Lianrui Fan, Bowen Ping +19Efficient VLM Inference Vision-Language Grounding
XPeng Inc. · Peking University · The University of Hong Kong +4
LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding
May 26, 2026Shihao Wang, Shilong Liu, Yuanguo Kuang +10Efficient VLM Inference Vision-Language Grounding
The Hong Kong Polytechnic University · NVIDIA · Princeton University +2
Angular Supervision for Visual Object Grounding in Autoregressive Vision-Language Models
Oct 23, 2026Parsa Esmaeilkhani, Longin Jan LateckiVision-Language Grounding VLM Interpretability
Temple University, Philadelphia, USA

Sep 30, 2026J/K move · Enter open · S save

GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

Qize Yu, Lianrui Fan, Bowen Ping, Xini Ding+18

XPeng Inc. · Peking University · The University of Hong Kong+4

Efficient VLM Inference Vision-Language Grounding Parallel Decoding Diffusion Language Models

Open paper AskPDF arXiv

Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a

4.51\times

speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.

Abstract

Vision-Language Grounding

Explore similar work

GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed

LocateAnything: Fast and High-Quality Vision-Language Grounding with Parallel Box Decoding

Angular Supervision for Visual Object Grounding in Autoregressive Vision-Language Models