Learning to Group and Order: Cross-Instance Self-Supervised RL for Vision-Centric MLLMs

Songming Yang; Hong-Tao Yu; Xiu-Shen Wei

cs.CVOct 31, 2026

Learning to Group and Order: Cross-Instance Self-Supervised RL for Vision-Centric MLLMs

Authors: Songming Yang, Hong-Tao Yu, Xiu-Shen Wei

Organizations: School of Computer Science and Engineering, Southeast University · Key Laboratory of New Generation Artificial Intelligence Technology and Its Interdisciplinary Applications, Southeast University

Abstract

Reinforcement learning (RL) post-training has become an important paradigm for improving multimodal large language models (MLLMs), especially with automatically verifiable rewards. Self-supervised visual pretext tasks provide such rewards without human annotations. However, existing methods mainly exploit within-instance structure, offering limited supervision for cross-instance discrimination and fine-grained comparison. We introduce Group-and-Order Self-Supervised Reinforcement Learning (GO-SSL), a cross-instance jigsaw framework for vision-centric MLLM post-training. Given two visual instances, GO-SSL mixes their local elements and trains the model to group them by instance of origin while recovering the order within each group, coupling instance-level comparison with spatial, temporal, or geometric structure recovery. This paired formulation turns the same unlabeled data into richer verifiable supervision through diverse pairings, shuffles, and a random-to-hard curriculum. We instantiate GO-SSL mainly on cross-image jigsaw tasks and further extend it to 3D depth and video temporal ordering. Across diverse benchmarks, GO-SSL consistently improves over the base MLLM and prior self-supervised post-training baselines, with clear gains in fine-grained perception and spatial understanding. Further evaluations across model scales and families support its generality. These results suggest that cross-instance comparison provides a useful self-supervised RL signal for improving transferable visual understanding in MLLMs. Code is available at https://github.com/SEU-VIPGroup/GO-SSL.

Figures & tables

Figure 1: GO-SSL: task construction and visual transfer. (a) Shuffled patches from two images are grouped by source and reordered to recover each image’s spatial layout. (b) A qualitative example of distinguishing visually similar object instances. (c) Category-wise benchmark averages for Qwen2.5-VL-7B and two GO-SSL variants, showing gains across all five categories.

Figure 2: Overview of GO-SSL. (a) Self-supervised task construction: local elements from paired images, RGB-D scenes, or videos are shuffled and assigned verifiable group-and-order targets. (b) RL post-training: the MLLM predicts source groups and their internal spatial, depth, or temporal order, and a deterministic verifier provides rewards.

Model	Omni3DBench	3DSRBench	BLINK	Static- EmbodiedBench	OmniSpatial	DA-2K	Avg.
	count	test	rd-acc	test	test	test
Qwen2.5-VL-7B	17.14	57.42	77.42	46.50	42.66	54.45	49.27
Spatial-SSRL [ 17 ]	20.00	58.21	83.87	48.00	44.29	59.28	52.28
Visual Jigsaw(3D) [ 18 ]	5.71	58.13	82.26	47.00	45.99	71.56	51.78
3D GO-SSL	24.29	58.35	91.13	49.00	46.18	74.90	57.31
(Gain)	+7.15	+0.93	+13.71	+2.50	+3.52	+20.45	+8.04

Table 2: Accuracy on 3D benchmarks. GO-SSL applies grouping and ordering to cross-scene depth.

Model	AoTBench	TUNABench	VideoMME	TempCompass		VSI-Bench	Video-TT	Avg.
	vqa	test	wo subs	mc	yorn	test	test
Qwen2.5-VL-7B	45.52	53.14	57.44	71.77	73.46	32.79	36.90	53.00
Visual Jigsaw(Video) [ 18 ]	51.67	55.10	58.07	73.10	74.68	35.39	38.90	55.27
Video GO-SSL	55.75	55.45	58.84	73.29	76.20	35.55	39.30	56.34
(Gain)	+10.23	+2.31	+1.40	+1.52	+2.74	+2.76	+2.40	+3.34

Table 3: Evaluation results on video benchmarks under the 16-frame setting. Video GO-SSL extends group-and-order learning to cross-video temporal ordering and improves temporal/video understanding over baselines.

Axis	Variant	Training setting	D&V	Spatial	Comp.	Gen.	Ext.	Avg.
Baseline	Qwen2.5-VL	–	70.41	55.40	61.59	55.42	31.71	56.75
	Spatial-SSRL	–	72.44	59.09	62.94	55.47	33.71	59.04
	Visual Jigsaw	–	74.08	58.17	63.02	56.08	32.77	59.13
Objective	Group only	Two images, 2 $\times$ 2; grouping only	71.08	55.66	62.12	55.52	32.07	57.14
	Order only	Single image, 2 $\times$ 2; ordering only	71.63	58.17	62.74	55.61	33.37	58.43
	G&O-Random †	Random pairs, 2 $\times$ 2; group + order	74.88	59.83	62.56	56.39	34.34	60.19

Table 4: Ablation studies of GO-SSL. Each trained variant is listed once. “D&V”, “Spatial”, “Comp.”, “Gen.”, and “Ext.” denote detail understanding and visual perception, spatial understanding and reasoning, compositional understanding, general multimodal benchmarks, and extra benchmarks, respectively. Avg. is computed over the same 14 benchmark metrics as Table 1 . Bold indicates the best result within each axis group.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Models	Reasoning	Benchmark						Avg.
Models	Reasoning	V*	VSR	OmniSpatial	SugarCrepe++	DA-2K	TUNABench	Avg.
3B Models
Qwen2.5-VL-3B	$\times$	75.39	77.27	45.08	61.26	52.03	50.35	60.23
Qwen2.5-VL-3B	$\checkmark$	67.54	68.34	40.05	50.49	54.16	48.11	54.78
7B Models
Qwen2.5-VL-7B	$\times$	76.96	77.31	42.66	61.59	54.45	53.14	61.02

Appendix

Table 5: Comparison of answer-only and reasoning-based evaluation prompting. Reasoning “

\times

” denotes direct answer-only prompting, while Reasoning “

\checkmark

” denotes CoT-style prompting. “Avg.” is computed over the listed benchmarks.

Method	Modality	Task form	Supervision multiplicity	Extra SFT	Tuning params	RL steps	Global batch	Rollout group
Spatial-SSRL [ 17 ]	Image / RGB-D	Multi-task single-source SSL	$\approx T\cdot n$ $(T=5)$	Yes	Full	360	128	5
Visual Jigsaw-Image [ 18 ]	Image	Single-source ordering	$n$	No	Full	1000	256	16
Visual Jigsaw-3D [ 18 ]	RGB-D	Single-source ordering	$n$	No	Full	800	128	16
Visual Jigsaw-Video [ 18 ]	Video	Single-source ordering	$n$	No	Full	1000	128	16
GO-SSL-Image	Image	cross-instance group + order	$\binom{n}{2}$	No	LoRA	1000	64	8
GO-SSL-3D	RGB-D	cross-instance group + order	$\binom{n}{2}$	No	LoRA	1000	64	8

Appendix

Table 6: Comparison of self-supervised RL post-training recipes. Supervision multiplicity follows the definition in Sec. 3.1 .

Model	D&V	Spatial	Comp.	General	Extra	Avg.
Qwen2.5-VL-3B
Qwen2.5-VL-3B	68.63	53.02	61.26	49.45	29.39	54.18
Spatial-SSRL-3B	70.08	55.66	60.94	50.22	32.33	56.05
GO-SSL-3B	70.78	56.37	61.31	50.69	32.51	56.62
LLaVA-OneVision-7B
LLaVA-OV-7B	66.83	51.79	56.74	56.42	23.88	53.12

Appendix

Table 7: Generalization across model scales and families. D&V denotes detail understanding and visual perception; Comp., General, and Extra denote the corresponding benchmark categories. LLaVA-OV abbreviates LLaVA-OneVision.

Model	Omni3D Bench	3DSRBench	BLINK	Static- EmbodiedBench	Omni Spatial	DA-2K
LLaVA-OV-7B	56.00	57.42	62.10	36.50	44.36	55.03
LLaVA-OV-7B + GO-SSL-3D	58.67	58.81	67.74	39.50	45.27	63.15

Appendix

Table 8: 3D evaluation on LLaVA-OneVision-7B (LLaVA-OV-7B). GO-SSL improves all six benchmark scores, indicating cross-backbone transfer. The benchmark subsets and evaluation metrics are consistent with Table 2 .

Model	Training Domain	Image-side Benchmarks					3D-side Benchmarks
		HRBench4K	HRBench8K	V*	VSR	SugarCrepe++	DA-2K	BLINK	Omni3DBench
		test	test	test	test	test	test	rd-acc	count
Qwen2.5-VL-7B	–	68.50	65.38	76.96	77.31	61.59	54.45	77.42	17.14
Image GO-SSL	Image	71.50	69.88	81.68	80.42	63.17	61.70	81.45	21.43
(Gain)	Image	+3.00	+4.50	+4.72	+3.11	+1.58	+7.25	+4.03	+4.29
3D GO-SSL	3D	69.38	67.88	80.10	78.95	62.14	74.90	91.13	24.29

Appendix

Table 9: Cross-domain transfer evaluation of GO-SSL variants. Image GO-SSL and 3D GO-SSL are evaluated on both image-side and 3D-side benchmarks. The results show that each variant is strongest in its own training domain, while still transferring non-trivially to the other domain.

Figure 3: Additional ablation studies. (a) Reward weight ablation on the image-side GO-SSL objective. (b) Video GO-SSL under different input-frame settings. (c) Order-only versus full Group+Order variants for video and 3D GO-SSL.

Pairing Strategy	D&V	Spatial	Comp.	Gen.	Extra	Avg.
CLIP-near + DINO-far	74.47	62.26	63.17	56.20	33.54	60.85
DINO-near + MAE-far	74.22	61.12	62.69	56.43	34.38	60.49
Difference	-0.25	-1.14	-0.48	+0.23	+0.84	-0.36

Appendix

Table 10: Comparison of feature-space pairing strategies under identical 1,000-step training budgets. CLIP-near + DINO-far corresponds to CLIP-Blind-600 in the main experiments. Differences are computed as DINO-near + MAE-far minus CLIP-near + DINO-far.

Variant	Final 200 steps	D&V	Spatial	Comp.	Gen.	Extra	Avg.
CLIP-Blind-600	$2\times 2$	74.47	62.26	63.17	56.20	33.54	60.85
CLIP-Blind- $2\times 3$	$2\times 3$	75.94	61.15	61.99	56.39	34.15	60.90
CLIP-Blind- $3\times 3$	$3\times 3$	76.19	61.11	61.95	56.53	34.26	60.99

Appendix

Table 11: Effect of progressively finer image layouts. All variants share 400 random

2\times 2

steps followed by 400 CLIP-Blind

2\times 2

steps; only the layout of the final 200 CLIP-Blind steps differs. Avg. is computed over 14 benchmarks.

Model	OC-Cpr	OC-Grp	VAW	State	StateAttr	Vis. Corr.
Qwen2.5-VL-7B	62.22	54.00	27.94	12.12	22.77	30.81
Visual Jigsaw	66.67	54.00	61.76	72.73	65.35	41.28
GO-SSL	71.94	62.00	75.00	84.85	78.22	43.60

Appendix

Table 12: Targeted evaluation of cross-instance discrimination. GO-SSL denotes the CB-2

\times

3 variant in Table 1 . All values are accuracies (%).

Figure 4: GO-SSL task examples across image, 3D, and video domains.

Figure 5: Qualitative comparison of occlusion-based sensitivity maps on fine-grained spatial matching examples. Compared with Qwen2.5-VL-7B and Visual Jigsaw, GO-SSL better localizes the visual evidence relevant to the queried description and produces correct answers. It also corrects a failure case of Spatial-SSRL in the second example, where object saliency alone is insufficient for precise spatial-relation judgment.

Figure 6: Qualitative examples on image-side tasks. GO-SSL gives correct predictions on perspective taking, dynamic reasoning, spatial interaction, and complex logic, where the base model and prior self-supervised post-training baselines often fail.

Figure 7: Qualitative examples on 3D-oriented visual understanding tasks. 3D GO-SSL gives correct predictions on physical contact reasoning, object counting, and relative depth estimation, where the base model and prior self-supervised post-training baselines often fail.

Figure 8: Qualitative examples on video understanding tasks. Video GO-SSL correctly answers questions requiring action intention reasoning, final-state tracking, and dynamic scene understanding, while the base model and video jigsaw baseline fail in these cases.

Panel of Figure 9 — Figure 9: Qualitative examples of visually grounded reasoning across diverse scenarios. Compared with the base model and prior self-supervised post-training baselines, GO-SSL gives more reliable answers on instance-level recognition, spatial reasoning, and video understanding, suggesting benefits beyond low-level spatial matching.

RL for VLMs

Reinforcement learning for vision-language models aligns multimodal reasoning, tool use, and action with task-specific feedback.

See all 488 papers

SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models
Apr 22, 2026Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr +2Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards
Max Planck Institute for Informatics, SIC · VIA Research Center · Google
V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance
Apr 14, 2026Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc +2Visual Instruction Tuning Large Vision-Language Models
Sorbonne Universite, CNRS, ISIR, F-75005 Paris, France · Institut universitaire de France (IUF)
PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions
Oct 27, 2026Tongkun Guan, Zhibo Yang, Mingkun Yang +3Vision-Language Models RL for VLMs
MoE Key Laboratory of Artificial Intelligence, AI Institute, School of Computer Science, Shanghai Jiao Tong University, Shanghai 200240, China · Qwen

Apr 22, 2026J/K move · Enter open · S save

SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models

Jiahao Xie, Alessio Tonioni, Nathalie Rauschmayr, Federico Tombari+1

Max Planck Institute for Informatics, SIC · VIA Research Center · Google

Multimodal Large Language Models Reinforcement Learning with Verifiable Rewards RL for VLMs Multimodal SSL

Open paper AskPDF arXiv

Reinforcement learning (RL) with verifiable rewards (RLVR) has demonstrated the great potential of enhancing the reasoning abilities in multimodal large language models (MLLMs). However, the reliance on language-centric priors and expensive manual annotations prevents MLLMs' intrinsic visual understanding and scalable reward designs. In this work, we introduce SSL-R1, a generic self-supervised RL framework that derives verifiable rewards directly from images. To this end, we revisit self-supervised learning (SSL) in visual domains and reformulate widely-used SSL tasks into a set of verifiable visual puzzles for RL post-training, requiring neither human nor external model supervision. Training MLLMs on these tasks substantially improves their performance on multimodal understanding and reasoning benchmarks, highlighting the potential of leveraging vision-centric self-supervised tasks for MLLM post-training. We think this work will provide useful experience in devising effective self-supervised verifiable rewards to enable RL at scale. Project page: https://github.com/Jiahao000/SSL-R1.

Abstract

RL for VLMs

Explore similar work

SSL-R1: Self-Supervised Visual Reinforcement Post-Training for Multimodal Large Language Models

V-GIFT: Boosting Visual Instruction Tuning with Self-Supervised Guidance

PerceptRL: Making Visual Perception Rewardable in MLLMs from Answers to Interactions