SL-Ground: Sentence-Level Grounding for Fine-Grained Open-Vocabulary Object Detection

Yang-Syu Li; Cheng-Huan Yang; Chia-Yung Jui; Peggy Joy Lu

cs.CVOct 17, 2026

SL-Ground: Sentence-Level Grounding for Fine-Grained Open-Vocabulary Object Detection

Authors: Yang-Syu Li, Cheng-Huan Yang, Chia-Yung Jui, Peggy Joy Lu

Organizations: Department of Computer Science and Information Engineering, National Chung Cheng University, Chiayi, Taiwan. · National Center for High-Performance Computing, Taichung, Taiwan.

Abstract

Grounding-based open-vocabulary detectors provide a widely used framework for text-conditioned localization, with a phrase-grounding formulation that aligns object queries to individual text tokens and recovers class scores through token-to-class aggregation. We observe that this token-level scoring design becomes a bottleneck for fine-grained open-vocabulary object detection, where the target category is defined by a complete compositional description rather than by isolated tokens. Based on this observation, we propose SL-Ground, a sentence-level grounding reformulation for grounding-based open-vocabulary detection. Instead of scoring queries against fragmented token evidence, SL-Ground encodes each category description as a complete sentence-level representation and directly scores object queries against these representations. To evaluate fine-grained recognition across label granularity, we construct a multi-granularity Stanford Cars benchmark spanning vehicle type, brand, and car model. At the finest car-model granularity, SL-Ground raises AP over the token-level variant from 14.5 to 70.6 under matched training candidate sampling and to 86.0 with full-vocabulary training, while better preserving open-vocabulary capability and avoiding repeated prompt chunking. Beyond Stanford Cars, SL-Ground also improves FG-OVD performance, achieving the best average AP among compared methods. These results show that sentence-level grounding provides a more suitable scoring unit for fine-grained OVOD and offers a practical path toward fine-grained grounding-based open-vocabulary detectors.

Figures & tables

Fig. 1: Example of compositional fine-grained labels and token sharing in vehicle categories.

Fig. 2: Overview of SL-Ground. Each class description is independently encoded into one CLIP sentence feature. The feature enhancer fuses visual and class features, class-level proposal scores initialize the decoder, and the prediction outputs boxes and query-to-class logits.

Detector setting			Label granularity (AP $\uparrow$ )
Model	Text encoder	Grounding unit	Type	Brand	Car model
GD-BERT	BERT	Token	70.3	40.1	10.3
GD-CLIP	CLIP	Token	80.5	73.3	14.5
SL-Ground	CLIP	Sentence	88.0	92.8	86.0

TABLE I: Stanford Cars detection results across label granularities.

Panel of Figure 4 — Fig. 3: Epoch-wise Stanford Cars AP over 20 fine-tuning epochs. SL-Ground stays above the token-level variants throughout, with the largest gap at car model granularity.

Fine-tuning setting		Stanford Cars AP $\uparrow$			OVOD AP $\uparrow$
Fine-tune target	Model	Type	Brand	Car model	COCO	miniLVIS
Zero-shot	GD-BERT	9.6	1.0	0.3	48.4	15.78
	GD-CLIP	1.6	0.6	0.1	45.6	26.37
	SL-Ground	1.4	0.6	0.9	48.8	25.27
Type	GD-BERT	70.3	3.0	2.6	15.8	6.01
	GD-CLIP	80.5	3.6	5.1	21.2	11.36

TABLE II: Cross-granularity transfer and OVOD knowledge preservation after Stanford Cars fine-tuning. Rows indicate the fine-tuning target, and COCO/miniLVIS AP measure retained open-vocabulary performance.

Method	Pretrain	Data (M)	AP $\uparrow$
GLIP [ 1 ]	O+G+C4 †	5.43	46.3
Grounding DINO-T(3) [ 2 ]	O+G+C4 †	5.43	48.4
YOLO-World-L [ 14 ]	O+G+C †	1.63	45.1
Dynamic-DINO [ 41 ]	O+G+V †	1.56	46.2
GLIP-T(C) [ 1 ]	O+G †	1.43	46.7
Grounding DINO-T(2) [ 2 ]	O+G †	1.43	48.1

TABLE III: COCO open-vocabulary detection results.

Method	AP $\uparrow$	AP r $\uparrow$	AP c $\uparrow$	AP f $\uparrow$
GLIP-T(A) [ 1 ]	18.5	14.2	13.9	23.4
GLIP-T(B) [ 1 ]	17.8	13.5	12.8	22.2
YOLO-World-L [ 14 ]	23.5	16.2	21.1	27.0
Grounding DINO-T [ 2 ]	16.0	5.0	10.0	23.0
OV-DINO [ 15 ]	24.4	15.5	20.2	29.7
SL-Ground	25.27	18.52	21.84	29.52

TABLE IV: miniLVIS open-vocabulary detection results.

Detector	Hard	Med.	Easy	Triv.	Col.	Mat.	Pat.	Trans.	Avg.
G-DINO	17.0	28.4	31.0	62.5	41.4	30.3	31.0	26.2	33.5
+ Finetune	16.6	28.0	24.3	73.6	42.7	29.4	29.3	28.4	34.0
+ HA-FGOVD	19.2	32.3	34.0	62.2	41.5	33.0	32.1	29.2	35.4
+ GUIDED	35.1	49.3	52.8	57.1	49.7	57.0	26.4	39.6	45.9
+ DSAA	38.5	53.0	59.2	63.3	56.0	52.3	46.5	63.0	54.0
SL-Ground	9.4	14.3	13.8	42.7	18.6	16.4	13.9	14.6	18.0

TABLE V: Comparison on FG-OVD benchmark.

Model	Dog-120	CUB-200	Car-196
LLaVA 1.5 [ 46 ]	38.96	35.24	46.92
LLaVA-NeXT (Mistral) [ 47 ]	38.86	34.88	49.48
MobileVLM v2 [ 48 ]	39.92	33.90	46.29
InstructBLIP Vicuna [ 49 ]	41.60	32.78	48.25
InstructBLIP Flan-T5-XL [ 49 ]	47.10	32.15	64.58
Phi-3-Vision [ 50 ]	39.80	37.63	54.50

TABLE VI: Fine-grained recognition results on Dog-120, CUB-200, and Car-196. Values are accuracy (%), not detection AP.

Model	Training setting	$K_{\mathrm{train}}$	AP $\uparrow$
GD-CLIP	75-token sampling	11.87 ∗	14.5
SL-Ground	75-token sampling	11.87 ∗	70.6
SL-Ground	Fixed size	12	65.5
SL-Ground	Fixed size	32	76.7
SL-Ground	Fixed size	64	83.0
SL-Ground	Full vocabulary	189	86.0

TABLE VII: Effect of training candidate-set size on Stanford Cars car-model detection. Evaluation uses all 189 categories.

Model	Params (M)	COCO			miniLVIS
Model	Params (M)	AP $\uparrow$	FPS $\uparrow$	Mem. $\downarrow$	AP $\uparrow$	FPS $\uparrow$	Mem. $\downarrow$
GD-BERT	173.0	48.4	12.38	1.37	15.78	0.58	1.34
GD-CLIP	187.1	45.6	6.28	1.27	26.37	0.26	1.25
SL-Ground	182.4	48.8	16.01	1.25	25.27	12.28	1.40

TABLE VIII: Accuracy–efficiency comparison on COCO and miniLVIS. Mem. denotes peak allocated GPU memory (GB).

Panel of Figure 12 — Fig. 4: Stanford Cars margin CDF and vocabulary-scaling latency. Chunk labels denote the number of chunks in the tokenized prompt.

Panel of Figure 13 — Fig. 5: Stage-level latency on Stanford Cars type labels, where all variants use one prompt chunk. Panel (a) shows stage composition, and panel (b) shows module-level latency savings relative to GD-BERT.

Panel of Figure 14 — Fig. 6: Controlled memory–latency trade-off with synthetic single-token labels. Token-level variants cap per-pass CUDA memory after chunking but incur repeated detector passes, whereas SL-Ground keeps one pass with lower latency and class-count-dependent memory growth.

Panel of Figure 15 — Fig. 7: Qualitative comparison on COCO. Grounding DINO is shown in the top row and SL-Ground in the bottom row. SL-Ground reduces class confusion in examples such as skateboard, bird, and horse while preserving comparable box locations.

Panel of Figure 16 — Fig. 8: Qualitative comparisons on miniLVIS with 1,203 candidate categories. Grounding DINO is shown in the top row and SL-Ground in the bottom row. Grounding DINO shows overlapping predictions with different class labels after outputs from multiple prompt chunks are concatenated without cross-chunk suppression, while SL-Ground produces a less dense set of predictions in a single pass.

Open-Vocabulary Object Detection

Open-vocabulary object detection lets models locate and label objects using text categories beyond their fixed training labels.

See all 89 papers

GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection
Mar 27, 2026Jiaming Li, Zhijia Liang, Weikai Chen +2Open-Vocabulary Object Detection Fine-Grained Visual Perception
School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Meituan, Shenzhen, China · GuangDong Province Key Laboratory of Information Security Technology, China +1
Fine-Grained Open-Vocabulary Object Detection with Fined-Grained Prompts: Task, Dataset and Benchmark
Mar 19, 2025Ying Liu, Yijing Hua, Haojiang Chai +2Open-Vocabulary Object Detection Object Detection
department of software engineering, Northeastern University, China. · His work was done in Articul8 AI.
Prompt Sensitivity in Vision-Language Grounding: How Small Changes in Wording Affect Object Detection
Apr 18, 2026Dawar Jyoti Deka, Amit Sethi, Syed Mohammad AliOpen-Vocabulary Object Detection VLM Robustness
Department of Aerospace Engineering, Indian Institute of Technology Bombay, Mumbai, India · Department of Electrical Engineering, Indian Institute of Technology Bombay, Mumbai, India

Mar 27, 2026J/K move · Enter open · S save

GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection

Jiaming Li, Zhijia Liang, Weikai Chen, Lin Ma+1

School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China · Meituan, Shenzhen, China · GuangDong Province Key Laboratory of Information Security Technology, China+1

Open-Vocabulary Object Detection Fine-Grained Visual Perception

Open paper AskPDF arXiv

Fine-grained open-vocabulary object detection (FG-OVD) aims to detect novel object categories described by attribute-rich texts. While existing open-vocabulary detectors show promise at the base-category level, they underperform in fine-grained settings due to the semantic entanglement of subjects and attributes in pretrained vision-language model (VLM) embeddings -- leading to over-representation of attributes, mislocalization, and semantic drift in embedding space. We propose GUIDED, a decomposition framework specifically designed to address the semantic entanglement between subjects and attributes in fine-grained prompts. By separating object localization and fine-grained recognition into distinct pathways, HUIDED aligns each subtask with the module best suited for its respective roles. Specifically, given a fine-grained class name, we first use a language model to extract a coarse-grained subject and its descriptive attributes. Then the detector is guided solely by the subject embedding, ensuring stable localization unaffected by irrelevant or overrepresented attributes. To selectively retain helpful attributes, we introduce an attribute embedding fusion module that incorporates attribute information into detection queries in an attention-based manner. This mitigates over-representation while preserving discriminative power. Finally, a region-level attribute discrimination module compares each detected region against full fine-grained class names using a refined vision-language model with a projection head for improved alignment. Extensive experiments on FG-OVD and 3F-OVD benchmarks show that GUIDED achieves new state-of-the-art results, demonstrating the benefits of disentangled modeling and modular optimization. Our code will be released at https://github.com/lijm48/GUIDED.

Abstract

Open-Vocabulary Object Detection

Explore similar work

GUIDED: Granular Understanding via Identification, Detection, and Discrimination for Fine-Grained Open-Vocabulary Object Detection

Fine-Grained Open-Vocabulary Object Detection with Fined-Grained Prompts: Task, Dataset and Benchmark

Prompt Sensitivity in Vision-Language Grounding: How Small Changes in Wording Affect Object Detection