Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design

Haoxiang Sun; Tao Wang; Chenwei Tang; Li Yuan; Jiancheng Lv

cs.CVFeb 25, 2026

Dr. Seg: Revisiting GRPO Training for Visual Large Language Models through Perception-Oriented Design

Authors: Haoxiang Sun, Tao Wang, Chenwei Tang, Li Yuan, Jiancheng Lv

Organizations: College of Computer Science, Sichuan University · School of Electronic and Computer Engineering, Peking University

Abstract

Following the success of Group Relative Policy Optimization (GRPO) in foundation LLMs, an increasing number of works have sought to adapt GRPO to Visual Large Language Models (VLLMs) for visual perception tasks (e.g., detection and segmentation). However, much of this line of research rests on a long-standing yet unexamined assumption: training paradigms developed for language reasoning can be transferred seamlessly to visual perception. Our experiments show that this assumption is not valid, revealing intrinsic differences between reasoning-oriented and perception-oriented settings. Using reasoning segmentation as a representative case, we surface two overlooked factors: (i) the need for a broader output space, and (ii) the importance of fine-grained, stable rewards. Building on these observations, we propose Dr.~Seg, a simple, plug-and-play GRPO-based framework consisting of a Look-to-Confirm mechanism and a Distribution-Ranked Reward module, requiring no architectural modifications and integrating seamlessly with existing GRPO-based VLLMs. Extensive experiments demonstrate that Dr.~Seg improves performance in complex visual scenarios while maintaining strong generalization. Code, models, and datasets are available at https://github.com/eVI-group-SCU/Dr-Seg.

Figures & tables

Figure 1: Dr. Seg achieves new state-of-the-art results on 5 out of 6 benchmarks under both in-distribution (ID) and out-of-distribution (OOD) conditions, demonstrating strong generalization ability.

Figure 2: Illustration of breadth-oriented exploration in perception-oriented tasks. Multiple level visual attributes can support different reasoning trajectories.

Figure 3: Comparison of token-level entropy during training between baseline VisionReasoner [ 25 ] model and baseline with a Look-to-Confirm strategy introduced in the next section. The numbers in parentheses are the performance on ReasonSeg dataset.

Figure 4: PCA visualization of last-token embeddings for baseline VisionReasoner [ 25 ] model vs. baseline with Look-to-Confirm. Points are 2D projections; shaded ellipses denote $2\sigma$ covariance contours around each group mean.

Method	RCO	RCO+	RCOg	ReasonSeg
Baseline	79.0	75.3	72.5	65.5
Raw reward	80.0	76.3	74.0	64.7

Table 1: Evaluation metrics of the baseline and the raw reward setting. Although the model achieves improved performance on the simpler RefCOCO, RefCOCO+, and RefCOCOg benchmarks, its performance degrades on the more challenging ReasonSeg benchmark, which features a mixture of single- and multi-object cases.

Figure 5: Overall training pipeline of Dr. Seg.

Method	In Distribution				Out of Distribution				Total
	RCO	RCO+	RCOg	ID avg	ReasonSeg		COCONut	OOD avg	avg
	testA	testA	test	ID avg	val	test	val	OOD avg	avg
Task-specific Models
LAVT [ 43 ]	75.8	68.4	62.1	68.8	-	-	-	-
ReLA [ 19 ]	76.5	71.0	66.5	71.3	22.4	21.3	-	-
Base 7B Models

Table 2: Performance on in-distribution (ID) and out-of-distribution (OOD) benchmarks.

\dagger

indicates that the code was not open-sourced as of the submission deadline.

*

denotes results obtained from our independent experiments under the same experimental conditions. RCO, RCO+, and RCOg refer to RefCOCO, RefCOCO+, and RefCOCOg, respectively. - denotes not available.

Method	Detection	Counting
	COCO	Pixmo		Count
	val	val	test	test
Qwen2-VL [ 35 ]	28.3	29.9	48.0	76.5
Qwen2.5-VL [ 1 ]	29.2	63.3	67.9	76.0
VisionReasoner [ 25 ]	37.7	70.1	69.5	87.6
Dr. Seg (Ours)	40.1	74.6	72.4	91.5

Table 3: Performance on Detection and Counting tasks. We approximate COCO AP using the ratio of the bounding box area to the total image area to enable compatibility with COCOAPI.

LC	DR	RCO	RCO+	RCOg	ReasonSeg
		79.0	75.3	72.5	65.5
✓		79.0	75.1	72.1	66.1
	✓	80.1	76.6	73.9	65.5
✓	✓	80.2	76.8	74.2	67.8

Table 4: Ablation study on Look-to-Confirm and Distribution-Ranked Reward. LC and DR refer to Look-to-Confirm and Distribution-Ranked Reward, respectively.

Normalization	RCO	RCO+	RCOg	ReasonSeg
Raw Reward	78.7	75.0	71.5	64.4
Distribution-Ranked	80.2	76.8	74.2	67.8

Table 5: Ablation on normalization strategy.

Figure 7: A snapshot of sample training rewards. Our Distribution-Ranked Reward maps each reward component to an empirical quantile $\mathbf{q}=[q_{1},q_{2},q_{3}]\in[0,1]^{3}$ using a FIFO history queue, obtaining rank-based score that is scale-invariant and overcoming high-variance domination.

Queue Size	RCO	RCO+	RCOg	ReasonSeg
128 (1)	80.1	76.0	73.6	63.3
512 (4)	80.0	76.3	73.6	66.3
2048 (16)	80.2	76.8	74.2	67.8
8192 (64)	80.4	77.3	74.3	65.2

Table 6: Ablation on queue size (numbers in parentheses indicate past steps).

Figure 8: Supplementary illustration of the Dr. Seg architecture. The model first performs a Look-to-Confirm exploration, generating multiple reasoning trajectories that describe diverse visual cues. During inference, these trajectories are used to prompt SAM to produce mask proposals. The example shown here demonstrates successful segmentation of all six semitone keys (6/6 correct).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Method	[email protected]	[email protected]	[email protected]	F1
Qwen2-VL	43.5	72.1	41.3	52.5
Qwen2.5-VL	47.6	72.4	46.6	56.7
VisionReasoner	57.3	62.6	63.9	63.3
Dr. Seg	59.0	69.6	64.3	66.8

Appendix

Table 7: More detection metrics on COCO.

Method	Length	gIoU
Baseline	77.8	65.5
+ Chain-of-Thought prompt	117.0	63.1
+ Look-to-Confirm prompt	137.3	59.6
Dr. Seg	148.3	67.8

Appendix

Table 8: Ablation on response length using test-time prompts.

Keyword	$R_{\text{format}}$	Length	gIoU
Random	$\checkmark$	89.6	64.9
Wait	$\checkmark$	95.3	65.6
Look ∗		73.6	65.8
Look	$\checkmark$	148.3	67.8

Appendix

Table 9: Ablation on response structure by retraining with/without the format reward

R_{\text{format}}

and varying the tag keyword. Look ∗ : without

R_{\text{format}}

, the model fails to stably generate

\langle\text{look}\rangle

and degrades toward the baseline.

Setting	RefCOCO testA	ReasonSeg val
$w$	35.6	24.1
$w\,x_{1}$	72.9	57.8
$w\,x_{1},x_{2}$	79.8	58.3
$w\,x_{1},x_{2},x_{3}$	80.2	67.8

Appendix

Table 10: Ablation on

R_{\text{accuracy}}

Dataset	Split	# of samples
DET	COCO	36,781
REC	RefCOCO	5,786
	RefCOCO+	5,060
	RefCOCOg	7,596
SEG	RefCOCO	1,975
	RefCOCO+	1,975

Appendix

Table 11: Statistics of evaluation benchmarks. Numbers combine validation and test splits where applicable. DET, REC, SEG, and CNT denote Detection, Referring Expression Comprehension, Referring Expression Segmentation, and Counting, respectively.

Task-specific Models
Method	Referring Expression Comprehension
	RefCOCO		RefCOCO+		RefCOCOg		avg
	val	testA	val	testA	val	test	avg
DQ-DETR [ 22 ]	88.6	91.0	81.7	86.2	82.8	83.4	85.6
Grounding-DINO-T(ft) [ 23 ]	89.2	91.9	81.1	87.4	84.2	84.9	86.5
Grounding-DINO-L(ft) [ 23 ]	90.6	93.2	82.8	89.0	86.1	87.0	88.1

Appendix

Table 12: Performance on the referring expression comprehension task.

Figure 9: Object count distribution of the COCONut multi-object split.

Figure 10: Categories included in our COCONut dataset, all of which are subclasses of the original COCO label set.

Figure 11: Visualization of our COCONut dataset, illustrating diverse object categories and a wide range of object counts per image.

Figure 12: A representative failure case of prior methods. The model initially attends to the correct person but drifts away as the chain-of-thought unfolds, leading to over or under counting the target instances. We attribute this behavior to excessive noise in the reward design: collapsing raw rewards into binary signals discards information and introduces noise. This noise is then further amplified when heterogeneous metric scales are summed together.

Figure 13: Visualization of the proposed Dr. Seg compared with VisionReasoner.

Figure 14: More visualization of the proposed Dr. Seg compared with VisionReasoner.

Explore similar work

SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation
Jun 2, 2026Hyunwoong Kim, Seongeun Lee, Hannah Yun +2Vision-Language Models Group Relative Policy Optimization
DEEPNOID Inc., Seoul, South Korea
Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models
Apr 2, 2026Zekai Ye, Qiming Li, Xiaocheng Feng +6RL for VLMs Large Vision-Language Models
Harbin Institute of Technology · Peng Cheng Laboratory · Huawei Technologies Co., Ltd
SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages
Oct 7, 2026Raja Kumar, Rajat Koner, Ritwick Chaudhry +3VLM Reasoning RL for VLMs
University of Southern California · Amazon AGI

Jun 2, 2026J/K move · Enter open · S save

SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation

Hyunwoong Kim, Seongeun Lee, Hannah Yun, Junhyun Park+1

DEEPNOID Inc., Seoul, South Korea

Vision-Language Models Group Relative Policy Optimization Credit Assignment RL for VLMs

Open paper AskPDF arXiv

Group Relative Policy Optimization (GRPO) and its variants, originally developed for Large Language Models (LLMs), have recently been applied to Multimodal LLMs and produced strong results. However, their coarse-grained holistic credit assignment from a single scalar advantage underfits vision-language (VL) tasks, where outputs are often long-form responses grounded in semantically rich images. To address this limitation, we exploit a structured signal that single-scalar formulations discard: the natural segmentation of long-form VL outputs. Concretely, we propose Segment-Decomposed GRPO (SD-GRPO), which z-normalizes verifiable per-segment rewards across the rollout group, yielding a vector of per-segment advantages in place of a single scalar. We evaluate SD-GRPO across three settings spanning controlled and real-world long-form VL generation, organized by increasing semantic entanglement across segments. On a controlled multi-panel dense-captioning task constructed from DOCCI, where segments are semantically independent, SD-GRPO consistently outperforms the GRPO baseline, with larger gains at higher segment counts. Extending to a controlled multi-chart long-form VQA task constructed from MultiChartQA, we show both theoretically and empirically that rollout-level rewards suffer from cross-segment credit misattribution that scales with output length. On a real-world scientific figure captioning task on the MMSci dataset, where subfigure captions share context across the figure, blending holistic and per-segment rewards further improves on both, suggesting per-segment normalization alone is insufficient when segments are semantically entangled. Finally, by integrating SD-GRPO into Dr. GRPO, we confirm that it can be applied to any GRPO framework with minimal implementation overhead to enhance long-form VL generation.

Abstract

Explore similar work

SD-GRPO: Verifiable Segment Decomposition for Long-Form Vision-Language Generation

Not All Tokens See Equally: Perception-Grounded Policy Optimization for Large Vision-Language Models

SPLIT-RL: Staged Perception-Language Reasoning Training with Claim-Level Advantages