Groundbench: Multi-Resolution Polygon Grounding Exposes the Geometry Gap in Vision-Language Models

Zhonghan Bian; Zhenran Wang; Jinsong Li; Zhangyang Qi

cs.CVAug 18, 2026

Groundbench: Multi-Resolution Polygon Grounding Exposes the Geometry Gap in Vision-Language Models

Authors: Zhonghan Bian, Zhenran Wang, Jinsong Li, Zhangyang Qi

Organizations: Equal contribution.

Abstract

Bounding-box scores on RefCOCO-family grounding leave little room to distinguish frontier vision-language systems, yet boxes discard object shape. We introduce GroundingBench, a matched benchmark that re-targets the same 1,500 image-expression-referent triples to exact-N polygons at five vertex budgets. A fixed-denominator harness separately audits filled-region intersection over union (IoU) and legal-polygon completion. The strongest tested configuration reaches 88.2 box IoU and 97.1 accuracy at IoU >= .5 ([email protected]), versus 57.7 and 69.2 for direct polygons; because these headline scores use different references, we also compare direct polygons with predicted boxes rasterised against the same contour target, obtaining 57.7 versus 57.3 when pooled. Performance is non-monotone in N and collapses at the densest budget, where legality failures compound residual geometric error. Qwen's thinking-setting contrast is the largest tested input-preserving configuration difference; under frozen templates, false spatial cues are more damaging than false colour cues, and target preference can remain high while contour tracing is poor. Alternate masks and a continuous-area scorer preserve the principal ordering. GroundingBench therefore measures an operational output-geometry gap spanning localisation, boundary construction, serialisation, and topology, rather than latent boundary perception alone.

Figures & tables

Figure 1: GroundBench freezes each image–expression–referent triple, derives five exact-budget targets, and scores exact-length outputs with a deterministic rasteriser.

$N$	Configuration	$\mathbf{IoU}^{\mathbf{2}}$	IoU p	IoU n	$\mathrm{Acc}@.1$	$\mathrm{Acc}@.3$	$\mathrm{Acc}@.5$	$\mathrm{Acc}@.7$	$\mathrm{Acc}@.9$
$N=8$	Doubao Seed-2.0-Pro	41.2/41.2	41.9/41.9	47.3/47.3	75.0/75.0	59.3/59.3	43.3/43.3	24.2/24.2	1.7/1.7
	Kimi K2.7 Code	38.2/38.2	38.6/38.6	43.6/43.6	83.8 /83.8	63.1/63.1	33.8/33.8	10.1/10.1	0.1/0.1
	Qwen 3.7 Plus	48.2 /48.4	49.1 /49.2	55.4 /55.6	81.3/81.7	71.1 /71.4	56.0 /56.2	30.3 /30.4	1.9 /1.9
	Gemini 3.5 Flash	62.6 / 62.6	63.6 / 63.6	71.7 / 71.7	93.0 / 93.0	87.8 / 87.8	76.0 / 76.0	48.8 / 48.8	8.0 / 8.0
$N=16$	Doubao Seed-2.0-Pro	46.8/46.8	46.8/46.8	48.7/48.7	78.7/78.7	65.1/65.1	51.5/51.5	33.4 /33.4	3.5 /3.5
	Kimi K2.7 Code	37.3/37.3	37.3/37.3	38.8/38.8	82.3 /82.3	60.0/60.0	32.9/32.9	10.1/10.1	0.1/0.1

Table 1: GroundBench filled-region results (

0

–

100

). Every metric cell shows fixed / parseable-only scoring under IoU 2 . IoU p uses the source part and IoU n is fidelity-normalised. Pooled and Mean average budgets and configurations; bold/underline mark best/second.

Figure 2: Summary of Table 1 : fixed IoU 2 , Acc@.5, parseable-only IoU, and pooled threshold accuracy. — Figure 2: Summary of Table 1 : fixed IoU 2 , [email protected], parseable-only IoU, and pooled threshold accuracy.

Table 4

	$N{=}8$			$N{=}16$			$N{=}24$			$N{=}32$			$N{=}64$
Config	Sel	Con	IoU	Sel	Con	IoU	Sel	Con	IoU	Sel	Con	IoU	Sel	Con	IoU
Doubao	$76.9$	$53.9$	$\mathbf{41.2}$	$80.5$	$57.7$	$\mathbf{46.8}$	$78.7$	$54.5$	$\mathbf{43.4}$	$80.0$	$54.9$	$\mathbf{44.4}$	$71.2$	$52.0$	$\mathbf{37.6}$
Kimi	$85.5$	$44.6$	$\mathbf{38.2}$	$84.7$	$43.3$	$\mathbf{37.3}$	$83.5$	$42.1$	$\mathbf{35.9}$	$80.9$	$40.8$	$\mathbf{33.7}$	$60.7$	$38.5$	$\mathbf{24.4}$
Qwen-off	$60.9$	$33.8$	$\mathbf{20.5}$	$62.6$	$23.4$	$\mathbf{14.7}$	$68.7$	$24.2$	$\mathbf{17.0}$	$71.3$	$25.6$	$\mathbf{18.8}$	$3.8$	$25.8$	$\mathbf{1.0}$
Qwen-max	$85.2$	$57.5$	$\mathbf{48.2}$	$86.5$	$57.6$	$\mathbf{49.7}$	$85.7$	$57.4$	$\mathbf{49.1}$	$85.9$	$56.7$	$\mathbf{48.6}$	$70.5$	$54.1$	$\mathbf{38.5}$
Gemini	$94.2$	$68.1$	$\mathbf{62.6}$	$92.4$	$70.8$	$\mathbf{64.7}$	$93.4$	$71.3$	$\mathbf{65.9}$	$86.0$	$70.1$	$\mathbf{59.5}$	$62.2$	$67.7$	$\mathbf{41.9}$

Table 4: Selection/tracing by budget in the best-effort view. Sel is target preference, Con is target IoU conditional on Sel, and IoU uses the fixed denominator.

	Difficulty			Distr. $D$			Compl. $Q$			Scale $RS$			Semantic group
Cfg.	Ea	Me	Ha	$D1$	$D2$	$D3{+}$	$Q1$	$Q2$	$Q3$	$RS1$	$RS2$	$RS3$	An.	Food	Ind.	Out.	Per.	Veh.
Doubao	$50.7$	$43.0$	$35.4$	$45.9$	$40.5$	$39.4$	$46.5$	$44.5$	$35.9$	$37.0$	$42.9$	$49.8$	$42.8$	$41.9$	$39.9$	$37.6$	$\underline{46.7}$	$44.9$
Kimi	$42.5$	$33.7$	$26.8$	$36.3$	$33.1$	$31.0$	$39.6$	$33.8$	$26.5$	$29.3$	$34.2$	$39.6$	$32.3$	$34.1$	$35.0$	$30.6$	$33.9$	$36.5$
Qwen-off	$17.5$	$13.7$	$12.5$	$15.9$	$13.8$	$12.7$	$15.3$	$14.0$	$13.6$	$12.6$	$14.6$	$16.5$	$17.3$	$13.9$	$11.9$	$12.8$	$15.6$	$14.1$
Qwen-max	$\underline{55.6}$	$\underline{46.4}$	$\underline{39.8}$	$\underline{50.3}$	$\underline{45.7}$	$\underline{42.6}$	$\underline{52.2}$	$\underline{46.9}$	$\underline{39.6}$	$\underline{42.9}$	$\underline{47.3}$	$\underline{51.5}$	$\underline{46.7}$	$\underline{48.1}$	$\underline{45.5}$	$\underline{42.7}$	$46.4$	$\underline{50.3}$
Gemini	$\mathbf{65.1}$	$\mathbf{58.1}$	$\mathbf{51.0}$	$\mathbf{61.8}$	$\mathbf{57.6}$	$\mathbf{52.3}$	$\mathbf{63.4}$	$\mathbf{57.8}$	$\mathbf{50.2}$	$\mathbf{54.2}$	$\mathbf{58.8}$	$\mathbf{61.4}$	$\mathbf{63.4}$	$\mathbf{59.3}$	$\mathbf{53.6}$	$\mathbf{59.4}$	$\mathbf{57.4}$	$\mathbf{61.3}$

Table 5: Pooled fixed-IoU slices. Ea/Me/Ha are easy/medium/hard; An./Ind./Out./Per./Veh. are animal/indoor/outdoor/person/vehicle. Difficulty,

D

Q

, and

RS

use IoU 2 ; semantic groups use best-effort. Bold/underline mark best/second.

Figure 3: Controlled fixed-IoU ablations. Diamonds mark controls; circles mark matched-image controls.

Visual Grounding

Visual grounding connects language to the image regions, objects, or pixels that support a description, question, or action.

See all 153 papers

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning
Aug 25, 2026Shuaishuai Cao, Min Huang, Meng Tang +3Spatial Reasoning Benchmarks Geospatial Reasoning
Central South University · Jiangxi Normal University · Hohai University +2
IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
Jul 17, 2026Xiuyuan Zhu, Ke Lu, Hao Wu +4Cross-Modal Knowledge Distillation Multimodal Large Language Models
University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China +1
Where Grounding Accuracy Lives on the IoU Curve: Label-Free Inference-Time Boundary Refinement
Aug 20, 2026Bo MaVision-Language Grounding Visual Grounding
Auckland University of Technology

Aug 25, 2026J/K move · Enter open · S save

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning

Shuaishuai Cao, Min Huang, Meng Tang, Xuan Liu+2

Central South University · Jiangxi Normal University · Hohai University+2

Spatial Reasoning Benchmarks Geospatial Reasoning Referring Image Segmentation Reasoning Segmentation Remote Sensing Benchmarks Geospatial Reasoning Benchmarks

Open paper AskPDF arXiv

Referring segmentation in overhead imagery is inherently relational: a query may ask for the buildings north of the road or the pond closest to a residential area, so the correct referent can contain one object, several objects, or none. Existing benchmarks mainly score mask overlap, which cannot verify whether a model actually resolved the stated spatial relation. We introduce GeoRefer-Bench, a benchmark for verifiable geospatial referring segmentation. Each query is represented by an executable logical form over a metric scene graph, and predictions are evaluated with Exact Query Success (EQS), which is satisfied only when the returned instance set exactly matches the set denoted by the query. GeoRefer-Bench contains 700 whole 2048x2048 UAV scenes (2.94 Gpx) at 12.5 and 25 cm ground sampling distance, 26,217 instances, 142,796 spatial relations, and 20,916 executable queries spanning five reasoning levels. It further includes three paraphrases per query, 24.0% unanswerable queries, 2,477 counterfactual pairs, and five leakage-controlled evaluation splits. An independent audit re-derives object geometry, mask ownership, relation values, query execution, and split provenance, finding zero issues across all 700 scenes. Relation-blind strategies can retain non-trivial mIoU while achieving at most 22.7 EQS overall, showing that overlap alone does not certify relational grounding. Across fifteen current models, the strongest reaches 74.1 EQS but drops from 98.9 at level 1 to 60.5 at level 5, while ten models score below 5 EQS on two-hop queries. GeoRefer-Bench turns geospatial referring segmentation from mask matching into verifiable reference resolution.

Abstract

Visual Grounding

Explore similar work

GeoRefer-Bench: A Benchmark from Referring Pixels to Verifiable Geospatial Reasoning

IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models

Where Grounding Accuracy Lives on the IoU Curve: Label-Free Inference-Time Boundary Refinement