Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding

Xin Zhang; Haochen Wang; Beibei Lin; Zhaoxiang Zhang; Robby T. Tan; Jason Li; Dacheng Tao

cs.CVOct 21, 2026

Generalized CycleGRPO for Fine-Grained Multimodal Understanding and Grounding

Authors: Xin Zhang, Haochen Wang, Beibei Lin, Zhaoxiang Zhang, Robby T. Tan, Jason Li, Dacheng Tao

Abstract

Fine-grained multimodal intelligence requires models to both understand a designated region and localize it from language. Existing methods learn these directions with separate objectives and task-specific annotations. We present Generalized CycleGRPO, the first unified reinforcement learning framework coupling region understanding and localization across spatial, temporal, and spatio-temporal domains. Given a region, a multimodal large language model generates candidate descriptions and then localizes each description to reconstruct the input region. Reconstruction overlap provides a verifiable reward for captioning, while individual localization rewards improve the inverse mapping. A common region-space formulation extends this cycle from image masks and boxes to video intervals and timestamped spatial evidence, without reference captions or an external language-model judge during reinforcement learning. In videos, the cycle encourages descriptions that distinguish a target event from similar recurrences. We introduce DOVE (Distinctive Occurrence-level Video-caption Evaluation), a human-verified contrastive benchmark for this temporal distinctiveness. CycleGRPO improves SAMTok's overall gIoU from 57.5% to 68.2% on GroundingSuite and average gIoU from 71.2% to 78.7% on GRES. For temporal tasks, it achieves 51.9% tIoU on Charades and 33.3% Temporal Evidence Coverage on DOVE, compared with 45.6% and 14.8% for the Qwen3-VL initialization. These results establish region-text cycle consistency as a general self-evaluation principle for multimodal understanding and localization. Models, code, and DOVE are available at https://devinxzhang.github.io/Generalized-CycleGRPO-Page/.

Figures & tables

Figure 1: From spatial CycleGRPO to Generalized CycleGRPO. (A) Region-captioning reinforcement learning relies on an extra critic to score generated descriptions. (B) Referring-localization reinforcement learning receives a rule-based geometric reward but optimizes only the localization direction. (C) Our spatial CycleGRPO closes the region–caption–localization loop and uses the overlap of the recovered spatial region as a verifiable reward. (D) This work generalizes the same cycle over spatial, temporal, and spatio-temporal regions, unifying captioning and localization for images and videos without an external critic.

Figure 2: Improvements over the corresponding base models on spatial (left), temporal (middle), and spatio-temporal (right) benchmarks. Within each benchmark, bar lengths show gains in percentage points and labels give the base and CycleGRPO scores.

Figure 3: Overview of Generalized CycleGRPO. Given a visual input $X$ and a target region $\mathcal{R}$ in a unified region space, the captioning rollout samples $N$ candidate descriptions $C_{i}$ . Each candidate then conditions $K$ localization rollouts that reconstruct regions $\widehat{\mathcal{R}}_{i,k}$ ; their agreement with the target provides the cycle-consistency signal. Spatial masks or boxes, temporal intervals, and spatio-temporal timestamped boxes differ only in their region serialization and consistency function, allowing the same region–text–region optimization to operate across image and video tasks.

Figure 4: Construction and evaluation of DOVE. Candidate intervals are pre-labeled, human-reviewed, and converted into private temporal probes. A text-only judge then evaluates captions for specified intervals using Order, State, and Outcome probes.

Method	Size	Pos.	Neg.	Avg.
GPT-4o [ 92 ]	–	43.4	79.6	61.5
Gemini 2.5 Pro [ 93 ]	–	36.5	75.2	55.8
VP-SPHINX [ 94 ]	13B	26.3	71.6	49.0
DAM † [ 46 ]	3B	52.3	82.2	67.3
SAMTok [ 31 ]	4B	43.5	80.4	61.9
+ Ours	4B	52.4	83.2	67.8

Table 1: Results on two inverse spatial tasks. Left: mask-to-text accuracy on DLC-Bench [ 46 ] . Right: text-to-mask gIoU on GroundingSuite [ 31 ] . All values are percentages. † denotes training on the in-domain DAM-1.5M data.

Method	Size	Overall	Perception				Reasoning
Method	Size	Overall	Color	Shape	Texture	Material	Position	Non-Entity	Relation
GPT-4o [ 92 ]	–	53.5	34.8	65.3	48.3	52.8	57.8	60.2	61.4
o3 † [ 97 ]	–	61.3	58.0	70.3	55.2	63.9	54.7	49.2	71.3
Gemini-2.5-Pro † [ 93 ]	–	64.2	62.3	68.8	58.6	66.7	64.1	64.9	70.3
Qwen2.5-VL [ 4 ]	3B	34.4	29.0	25.0	34.5	30.6	43.8	26.2	44.6
Qwen2.5-VL [ 4 ]	7B	41.7	39.1	40.6	44.8	27.8	59.4	36.1	40.6

Table 2: Results on GAR-Bench-VQA [ 47 ] . We report accuracy (%) for fine-grained regional perception and reasoning. † denotes thinking mode.

Method	Size	METEOR	CIDEr	AP 50	mIoU	Recall
Method	Size	Val / Test	Val / Test	Val / Test	Val / Test	Val / Test
LISA [ 49 ]	7B	13.0 / 12.9	33.9 / 32.2	25.2 / 24.8	62.0 / 61.7	36.3 / 35.5
GLaMM [ 30 ]	7B	16.2 / 15.8	47.2 / 43.5	30.8 / 29.2	66.3 / 65.6	41.8 / 40.8
OMG-LLaVA [ 51 ]	7B	14.9 / 14.5	41.2 / 38.5	29.9 / 28.6	65.6 / 64.7	– / –
Sa2VA [ 34 ]	8B	16.4 / 16.2	49.5 / 49.0	33.2 / 32.2	67.7 / 66.8	45.1 / 44.5
SAMTok [ 31 ]	4B	16.1 / 16.4	48.2 / 51.4	34.7 / 34.4	69.4 / 68.4	46.6 / 48.3

Table 3: Results on interleaved text-mask generation on GCG [ 30 ] (left) and generalized referring expression segmentation on GRES [ 48 ] (right). GCG entries are Val / Test. For GRES, split entries are Val / Test A / Test B and averages are reported separately; N-acc denotes no-target accuracy.

Model	Size	Charades				ActivityNet				QVHighlights
Model	Size	[email protected]	[email protected]	[email protected]	tIoU	[email protected]	[email protected]	[email protected]	tIoU	[email protected]	[email protected]	[email protected]	tIoU
GPT-4o [ 92 ]	–	60.6	44.5	23.5	41.8	55.2	41.4	25.8	40.4	69.0	54.8	38.5	52.1
GPT-5 [ 104 ]	–	59.3	42.0	22.0	40.5	57.4	44.9	30.4	42.9	72.4	60.4	46.4	56.8
Gemini-2.5-Pro [ 105 ]	–	74.1	61.1	34.0	52.8	72.3	64.2	47.1	58.1	84.1	75.9	61.1	70.4
VideoChat-Flash [ 106 ]	7B	60.2	37.9	17.8	39.7	35.5	21.8	10.5	24.8	45.2	30.6	16.7	32.7
VideoChat-R1 [ 107 ]	7B	51.9	30.8	11.7	33.7	35.0	23.9	11.3	25.0	29.3	19.1	9.4	21.5

Table 4: One-to-one temporal grounding results on Charades-STA [ 87 ] , ActivityNet Captions [ 88 ] , and QVHighlights [ 89 ] . We report Recall@1 at temporal IoU thresholds of

0.3

0.5

, and

0.7

, together with temporal IoU (tIoU), averaged over examples.

Model	Size	Temp-Cov $\uparrow$	Clip-Complete $\uparrow$	Order $\uparrow$	State $\uparrow$	Outcome $\uparrow$	N/M $\downarrow$	TEC $\uparrow$
GPT-5.6-sol	–	17.3	5.9	16.4	17.9	17.7	71.1	17.3
TRACE-Uni	–	2.0	0.6	1.3	1.9	3.0	95.7	2.1
Sa2VA [ 34 ]	8B	0.0	0.0	0.0	0.0	0.0	99.9	0.0
VideoChat2-HD [ 109 ]	7B	1.8	0.4	1.7	1.7	2.1	93.5	1.8
OMTG [ 99 ]	4B	0.7	0.2	0.6	1.0	0.6	98.8	0.7
VideoChat-R1 [ 107 ]	7B	3.2	0.4	2.8	2.9	4.2	93.9	3.3

Table 5: Temporal captioning results on DOVE. Temp-Cov is the percentage of probes whose correct answers are supported by the caption; Clip-Complete is the percentage of intervals with all answers supported; and Order, State, and Outcome report type-specific coverage. N/M denotes unmentioned probes, while Temporal Evidence Coverage (TEC) is the macro-average of Order, State, and Outcome coverage. All values are percentages.

Method	DLC-Bench			GroundingSuite ([email protected])
Method	Pos.	Neg.	Avg.	Single	Stuff	Multi	Part	All
Qwen2.5-VL-3B (bbox)	19.4	44.8	32.1	93.7	32.8	63.2	66.3	69.0
+ Ours	23.3	46.6	35.0	94.1	34.1	64.1	67.3	69.9
Qwen3-VL-4B (bbox)	39.6	62.4	51.0	95.2	24.2	76.7	67.0	71.4
+ Ours	43.2	65.4	54.3	94.9	33.9	76.3	71.1	74.1
Qwen3-VL-8B (bbox)	41.0	65.4	53.2	94.9	31.2	71.8	67.4	71.4

Table 6: Ablation on spatial localization formats using continuous bounding-box coordinates. Results are reported on DLC-Bench [ 46 ] and GroundingSuite [ 31 ] ; all values are percentages.

Model	Size	What	When (Temporal IoU)		Where (Visual IoU)		Overall
Model	Size	Acc	Chain1	Chain2	Chain1	Chain2	mAM	mLGM
GPT-4o [ 92 ]	–	60.8	16.7	12.8	6.5	3.0	26.8	38.2
Gemini-2-Flash [ 100 ]	–	53.0	24.5	23.8	4.6	2.2	26.9	35.6
Video-LLaMA3 [ 111 ]	7B	41.9	23.0	23.1	0.9	0.2	21.7	27.0
LLaVA-Video [ 110 ]	7B	49.5	10.5	12.2	1.9	1.3	20.8	27.3
VideoChat2 [ 109 ]	7B	36.2	13.7	12.5	2.5	1.0	17.0	20.3

Table 7: Results on V-STaR [ 112 ] . What measures answer accuracy, When measures temporal IoU, and Where measures visual IoU. Chain1 denotes what–when–where and Chain2 denotes what–where–when; mAM and mLGM are the aggregate metrics.

Model	Size	VideoMME		WorldSense		VideoMMMU		TVGBench	Avg.
Model	Size	Overall	Long	Overall	Recognition	Overall	Perception	mIoU	Avg.
GPT-4o [ 92 ]	–	71.9	–	42.6	–	61.2	66.0	–	–
Video-LLaMA3 [ 111 ]	7B	60.6	48.7	37.3	38.1	46.5	59.7	22.2	44.7
InternVL-2.5 [ 114 ]	8B	62.3	51.2	39.6	38.5	42.4	57.0	6.3	42.5
Qwen2.5-VL (base) [ 4 ]	7B	62.4	50.8	36.1	33.7	51.2	64.7	16.3	45.0
VideoRFT [ 117 ]	7B	59.8	50.7	38.2	36.6	51.1	66.0	14.3	45.2

Table 8: Results across general video understanding, reasoning, and temporal grounding. VideoMME [ 84 ] , WorldSense [ 115 ] , and VideoMMMU [ 116 ] report accuracy, while TVGBench [ 108 ] reports mIoU. Avg. is the mean over the reported columns.

Figure 5: Region-conditioned captioning across spatial, temporal, and spatio-temporal regions. Blue marks targets, red marks errors or omissions, and green marks target-specific evidence from Generalized CycleGRPO.

Model	Caption Reward	Localization Reward	Charades				DOVE
Model	Caption Reward	Localization Reward	[email protected]	[email protected]	[email protected]	tIoU	TEC $\uparrow$	Temp-Cov $\uparrow$	Clip-Complete $\uparrow$	Order $\uparrow$	State $\uparrow$	Outcome $\uparrow$	N/M $\downarrow$
Baseline	–	–	72.0	58.3	32.0	50.5	14.8	14.7	5.5	12.6	17.1	14.7	75.1
+ Caption reward only	✓	✗	73.0	59.6	32.3	51.4	30.1	30.0	13.4	26.9	33.8	29.6	53.0
+ Localization reward only	✗	✓	71.5	58.5	32.4	51.0	29.6	29.7	13.8	29.0	32.1	27.5	52.9
+ Ours	✓	✓	72.9	60.2	32.7	51.8	33.3	33.4	14.6	31.8	37.4	30.8	50.7

Table 9: Reward-side ablation for temporal grounding on Charades-STA [ 87 ] with OMTG [ 99 ] , and temporal captioning on DOVE with Qwen3-VL [ 5 ] . All values are percentages.

Model	Charades				DOVE
Model	[email protected]	[email protected]	[email protected]	tIoU	TEC $\uparrow$	Temp-Cov $\uparrow$	Clip-Complete $\uparrow$	Order $\uparrow$	State $\uparrow$	Outcome $\uparrow$	N/M $\downarrow$
Qwen3-VL [ 5 ]	67.1	47.5	24.1	45.6	14.8	14.7	5.5	12.6	17.1	14.7	75.1
+ Captioning-only GRPO	69.9	54.2	28.6	49.0	17.0	17.0	6.6	13.4	22.9	14.7	74.9
+ Localization-only GRPO	74.4	58.0	29.8	51.2	29.4	29.3	13.6	25.8	33.3	29.0	52.9
+ Ours	74.8	58.6	30.7	51.9	33.3	33.4	14.6	31.8	37.4	30.8	50.7

Table 10: Training-objective ablation with Qwen3-VL [ 5 ] for temporal grounding on Charades-STA [ 87 ] and temporal captioning on DOVE. All values are percentages.

Model	V-STaR	VideoMME	WorldSense	VideoMMMU	TVGBench
	mLGM	Acc.	Acc.	Acc.	mIoU
Open-o3-Video [ 39 ]	47.2	61.0	39.3	58.0	21.1
+ Captioning-only GRPO	47.9	61.8	39.1	58.4	23.7
+ Localization-only GRPO	47.3	61.9	39.6	58.1	24.9
+ Ours	49.8	62.3	40.2	59.7	28.5

Table 11: Training-objective ablation with Open-o3-Video-4B [ 39 ] for spatio-temporal grounding on V-STaR [ 112 ] , video understanding on VideoMME [ 84 ] , WorldSense [ 115 ] , and VideoMMMU [ 116 ] , and temporal grounding on TVGBench [ 108 ] . All values are percentages.

Model	V-STaR		VideoMME	WorldSense
	mAM	mLGM	Overall	Overall
Open-o3-Video [ 39 ]	33.4	45.7	63.0	39.0
+ Ours	34.4	47.5	63.4	40.0

Table 12: Spatio-temporal grounding and general video understanding results with Open-o3-Video [ 39 ] based on Qwen2.5-VL-7B. V-STaR [ 112 ] reports mAM and mLGM; VideoMME [ 84 ] and WorldSense [ 115 ] report overall accuracy. All values are percentages.

Setting	Configuration
Initialization	Qwen3-VL-4B-SAMTok [ 31 ]
Training data	20K DenseWorld [ 119 ] images with object masks and 1K GRES [ 48 ] no-target expressions
Region representation	Discrete mask tokens in both prompts and responses
Input prompt	<image> “Provide a detailed description of this region <mask tokens> .” When a zoomed view is available, the prompt continues: “Zoom in with the perspective as <image> , <mask tokens> .”
Training objective	Spatial mask–text–mask CycleGRPO
Trainable modules	Language model and multimodal projector; visual encoder frozen

Table 13: Implementation details for the spatial setting. The configuration is carried over unchanged from the ECCV version [ 38 ] .

Setting	Qwen3-VL initialization	OMTG initialization
Initialization	Qwen3-VL-4B [ 5 ] with bidirectional co-SFT	OMTG-4B RL [ 99 ] with bidirectional co-SFT
RL training examples	18,742	18,735
Training data	OMTG [ 99 ] : video-disjoint RL split comprising the remaining 50% of source videos	OMTG [ 99 ] : filtered visually distinct temporal examples
Rollout / actor update batch size	128 / 128	64 / 64
Caption rollouts $N$	4	4
Localization rollouts $K$	4	4

Table 14: Training configurations for temporal CycleGRPO with Qwen3-VL [ 5 ] and OMTG [ 99 ] initializations.

Setting	Configuration
Initialization	Open-o3-Video-4B (RL) [ 39 ] , followed by one epoch of inverse-caption and video-QA co-SFT
Training data	Open-o3-Video [ 39 ] : 35,887 examples comprising 4,143 spatial, 4,820 temporal, 11,924 spatio-temporal, and 15,000 general video-QA examples
Region representations	Normalized image boxes, temporal intervals, and sparse timestamp–box supports
Training objectives	Region–text–region CycleGRPO for the three region tasks; answer-based rewards for auxiliary video QA
Trainable modules	Language model and multimodal projector; visual encoder frozen
Precision / memory	bfloat16 / gradient checkpointing

Table 15: Implementation details for the Open-o3-Video-4B mixed-task training used in the main spatio-temporal and general-video results.

Figure 6: Qualitative comparison of region captioning on DLC-Bench [ 46 ] between SAMTok [ 31 ] and CycleGRPO. Target regions are highlighted in green. Within the captions, green text marks accurate attributes and red text marks errors.

Figure 7: Qualitative mask-localization results on GRES [ 48 ] . CycleGRPO is compared with SAMTok [ 31 ] and the ground truth.

Figure 8: Qualitative mask-localization results on GroundingSuite [ 31 ] . The first row evaluates part-level segmentation; the second evaluates amorphous stuff regions. Predictions from SAMTok and CycleGRPO are compared with the ground truth.

Figure 9: Qualitative GroundingSuite results with continuous bounding-box outputs. CycleGRPO predictions, Qwen3-VL predictions, and ground-truth boxes are shown in red, blue, and green, respectively.

Figure 10: A spatial region–text–region rollout. Each of four descriptions of the target mushroom conditions six localization rollouts. Green and red text mark accurate and erroneous evidence; blue masks show the reconstructed regions. More precise descriptions generally yield higher reconstruction IoU.

Figure 11: Qualitative temporal grounding comparisons on TimeLens [ 33 ] . Blue-bordered frames depict the reference event, and the timelines compare the reference interval with predictions from Qwen3-VL-4B-Instruct, OMTG, and our model. The displayed IoU values denote temporal IoU.

Figure 12: Qualitative V-STaR [ 112 ] examples for joint temporal–spatial grounding. Each panel contains a query, sampled frames with the associated object evidence, and the predicted temporal interval. The green bar and shaded band denote the reference interval; blue and red bars denote Open-o3-Video and Generalized CycleGRPO, respectively.

Figure 13: Representative failure cases of Generalized CycleGRPO on V-STaR [ 112 ] . From upper left to lower right, the examples show incorrect temporal retrieval despite a correct semantic answer, imprecise spatial localization with an over-broad temporal prediction, semantic hallucination for an unusual interaction, and a missed event near the end of a long video. Green and red denote the ground truth and CycleGRPO predictions, respectively.

Figure 14: Statistics of DOVE. (A) Absolute source-video and target-interval duration distributions on a logarithmic time axis (seconds); boxes show the interquartile ranges and whiskers show the full ranges. (B) Number of same-query occurrences per group. (C) Query-domain distribution from a deterministic lexical taxonomy over the human-reviewed queries. (D) Most frequent non-actor query terms. (E) Human-verified temporal probes by evidence axis. (F) Model-independent difficulty indicators: groups with four or five occurrences, target intervals shorter than five seconds, and intervals covered by all three probe axes.

Multimodal Grounding

Multimodal grounding links language and other signals to the specific objects, regions, events, or actions that support an interpretation.

See all 185 papers

Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
Jul 13, 2026Xin Zhang, Haochen Wang, Yikang Zhou +3Group Relative Policy Optimization Multimodal Large Language Models
National University of Singapore · ByteDance
TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs
Jul 19, 2026Yuhan Zhu, Changlian Ma, Xiangyu Zeng +12Temporal Video Grounding Reinforcement Learning
Nanjing University · Shanghai AI Laboratory · Shanghai Jiao Tong University +3
Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification
Jun 27, 2026Tianshu Zhang, Yan Wang, Ji Qi +1Temporal Video Grounding Efficient Multimodal Inference
Tsinghua University

Jul 13, 2026J/K move · Enter open · S save

Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

Xin Zhang, Haochen Wang, Yikang Zhou, Zhuochen Wang+2

National University of Singapore · ByteDance

Group Relative Policy Optimization Multimodal Large Language Models Referring Image Segmentation Vision-Language Grounding RL for VLMs

Open paper AskPDF arXiv

This paper introduces Actor as Its Own Critic, a unified reinforcement learning framework, Cycle Group Relative Policy Optimization (CycleGRPO), that jointly optimizes region understanding and localization for Multimodal Large Language Models (MLLMs). Unlike existing separate pipelines, we leverage the inherent duality between the two tasks to construct a self-evaluating reinforcement learning paradigm: "region

\to

text

\to

region''. Specifically, a single MLLM first acts as the actor to generate region captions, then immediately transitions to a critic to ground its generated text back in the spatial domain. Therefore, CycleGRPO requires only region inputs, e.g., masks or bounding boxes, entirely bypassing the need for textual ground truths. A quality-aware token-level cycle-consistency reward is employed to assess the semantic discriminability of text captions via their physical localization accuracy. Empirically, built upon SAMTok, our CycleGRPO framework successfully bootstraps both capabilities simultaneously. Without any task-specific fine-tuning, the framework yields consistent performance gains across a wide range of benchmarks, including region captioning, region VQA, grounded dialogue, and referring segmentation. Overall, CycleGRPO offers a straightforward and scalable way to advance pixel-level capabilities in MLLMs. Code and models are released at https://github.com/devinxzhang/CycleGRPO.

Abstract

Multimodal Grounding

Explore similar work

Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification