Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

Qingan Liu; Juntong Feng; Yue Zhu; Yujie Cheng; Xinzhe Han; Shuai Hao; Yunzhi Zhuge; Haiwen Diao

cs.CVOct 13, 2026

Game2Real-Bench: Do Toy Games Reveal and Transfer to Real-World Visual Reasoning?

Authors: Qingan Liu, Juntong Feng, Yue Zhu, Yujie Cheng, Xinzhe Han, Shuai Hao, Yunzhi Zhuge, Haiwen Diao,

Organizations: Dalian University of Technology · Nanyang Technological University

Abstract

Visual reasoning is an important capability for vision-language models (VLMs) to understand visual evidence, spatial structure, constraints, and state changes. Existing evaluations mainly rely on real-world data or procedurally generated tasks. Real-world data preserve natural complexity, while procedural games provide controllable generation and verifiable solutions. However, it remains unclear whether abilities measured in toy games correspond to real-world visual reasoning and whether game examples can support transfer. We introduce Game2Real-Bench, a benchmark that studies these connections across four capability groups, including fine-grained perception, spatial reasoning, constraint reasoning, and sequential planning. We construct 1,000 procedurally generated game samples with executable verification and curate 400 real-world samples aligned by shared reasoning requirements rather than visual similarity. We evaluate 25 representative VLMs and observe strong game-to-real consistency, with Pearson and Spearman correlations of 0.935. We further study one-shot game-to-real transfer on four models. Constraint reasoning improves for several models, while fine-grained perception declines with game demonstrations. We also construct Game2Real-Dataset with nearly 24K verified game samples across 18 tasks and 65 QA families. Our benchmark, dataset, and code are available at https://github.com/QingAnLiu/Game2Real-Bench.

Figures & tables

Figure 1: Multi-stage construction pipeline of Game2Real-Bench. It comprises capability-guided task design, procedural generation and verification, and mechanism-aligned real-world curation.

Figure 2: Capability-aligned tasks in Game2Real-Bench. Toy-game and real-world tasks span fine-grained perception, spatial reasoning, constraint reasoning, and sequential planning. We present representative QA samples illustrating shared reasoning requirements across the two settings.

Model	Fine-grained Perception	Spatial Reasoning	Constraint Reasoning	Sequential Planning	Overall
$\blacktriangledown$ Proprietary Models
GPT-5.6-Sol	53.9 / 57.0	69.8 / 49.0	81.3 / 63.9	81.4 / 68.0	69.8
Gemini-3.1-Pro-Preview	36.0 / 66.0	59.0 / 55.0	56.7 / 51.1	75.2 / 73.0	59.6
Claude-Opus-4.8-thinking	22.5 / 50.0	50.5 / 44.0	62.0 / 46.8	70.8 / 61.0	53.7
$\blacktriangledown$ Large Open-Weight Models
Qwen3.8-27B	37.6 / 67.0	61.3 / 55.0	39.3 / 38.8	61.0 / 72.0	52.6

Table 1: Scores (%) on Game2Real-Bench, with a 1,000-sample Toy Game set and a 400-sample Real-World set. For each capability, results are reported as Toy Game / Real-World. Toy-game scores are accuracies; real-world scores incorporate SpatiaLQA F1 where applicable. Overall denotes the mean per-sample score over all 1,400 samples. Open-weight models are grouped by total parameter count as large (27–35B), medium (7–12B), and small (2–4B). For MoE models, active parameter counts are retained in the model names. The best Toy Game and Real-World results within each group are highlighted in bold ; models are ordered by Overall score.

Figure 3: Capability-wise game-to-real transfer with one-shot in-context learning. Compared with zero-shot evaluation, random game demonstrations improve constraint reasoning on average but reduce fine-grained perception accuracy for all four models on the 400 real-world targets.

Figure 6: Failure cases on Game2Real-Bench. Model reasoning traces are compared with verified solutions for Block Counting, Hexakai, and Mancala. These cases illustrate three recurring failures: misinterpreting spatial structure, inconsistent relational reasoning, and unreliable state updating.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

	Dataset		Evaluation
Task	Families	Samples	Open	Choice	Images
C1: Fine-grained Perception
Missing Tiles	3	518	45	18	1, 7
Hidden Object	4	720	47	29	1, 2
Dot Shape	3	480	0	24	7
Color Swap	4	600	0	15	7

Appendix

Table 2: Task inventory and sample counts. Dataset columns report QA families and samples in Game2Real-Dataset, while evaluation columns report open-ended and single-choice samples in the toy-game set. Images reports the number of visual inputs per evaluation sample, including separately provided answer candidates.

Task	QA-family coverage
C1: Fine-grained Perception
Missing Tiles	Complete connector patterns; side states; port lists and degrees.
Hidden Object	Whole-scene and region-specific counts across target classes.
Dot Shape	Contour matching; vertex counts; counts by contour layer.
Color Swap	Transformed models; full or partial color histograms; changed and unchanged piece counts.
C2: Spatial Reasoning

Appendix

Table 3: QA-family coverage by task. Each row summarizes the query targets represented in Game2Real-Dataset. The number of families per task is reported in Table 2 .

Option	Candidate grid	Relation to target	Valid
A	6 6 4 / 1 3 2 / 4 2 5	Exact arrangement	Yes
B	5 2 4 / 2 3 1 / 4 6 6	$180^{\circ}$ rotation	No
C	4 5 4 / 6 1 1 / 2 6 3	Final Bottom grid	No
D	4 2 5 / 6 3 2 / 6 1 4	$270^{\circ}$ rotation	No
E	4 1 6 / 2 3 6 / 5 2 4	$90^{\circ}$ rotation	No
F	4 6 6 / 2 3 1 / 5 2 4	Left–right reflection	No

Appendix

Table 4: Candidate verification for Rolling Cube. The options are taken from Game2Real-Dataset sample GBT-COT-004927 ; slashes separate grid rows. Rotations are clockwise relative to the recomputed Top grid. Only A agrees at every position under the specified reading convention.

Source	$N$	Construction
GQA ( Hudson and Manning, 2019 )	12	10 original QA; 2 lightly reformulated.
OmniSpatial ( Jia et al., 2026 )	61	Original visual and spatial reasoning QA.
SPair-71k ( Min et al., 2019 )	27	QA derived from keypoint correspondences.
Visual-Jigsaw ( Wu et al., 2026 )	40	Lightly reformulated completion questions.
CAPTURe-real ( Pothiraj et al., 2025 )	30	Lightly reformulated counting questions.
All-Angles-Bench ( Yeh et al., 2026 )	20	Original synchronized-view questions.

Appendix

Table 5: Real-world sources and construction. Counts refer to retained Game2Real-Bench samples. Original QA is kept where suitable; light reformulation preserves source answers, while annotation-derived QA introduces questions grounded in source annotations.

Pre-step frames		Post-step frames
Time (s)	Label	Time (s)	Label
191.60	F2	256.48	F8
192.60	F7	257.48	F1
193.60	F5	258.48	F3
194.60	F6	259.48	F4

Appendix

Table 6: Frame selection for the IKEA applaro example. Timestamps are measured in seconds from the beginning of the source video. Each group contains four frames; labels identify their shuffled positions after the instruction diagram in the final nine-image input.

Setting	Value	Setting	Value
Backend	vLLM	Temperature	0
Sampling	Disabled	Context limit	65,536
Top- $p$ / top- $k$	0.95 / 20	Presence penalty	1.5
Thinking budget	$32\mathrm{k}$ tokens	Answer budget	512 tokens
Custom prompts	Disabled	Answer stage	Conditional

Appendix

Table 7: Configuration of the controlled Qwen in-context learning runner. Paired zero-shot and one-shot runs share these inference settings. In thinking mode, the additional answer budget is used only when the initial budget is exhausted without a final answer.

Subset or answer type	Samples	Scoring criterion
Single-choice subsets	236	Exact option-label match.
GQA short answers	12	Normalized short-answer match.
CAPTURe-real counts	30	Exact count correctness.
VisualTrans and IKEA frame selection	8	Exact selected labels, ignoring order.
SSI-Bench rankings	50	Exact match of the full ranking.
ReasonMap routes	14	Match to an accepted structured route.

Appendix

Table 8: Scoring rules for the 400 real-world samples. Counts refer to the final selected set. All subsets use binary correctness except SpatiaLQA, which contributes a normalized graph score.

Toy task	Toy	Real	Real-world sources	Pool
C1: Fine-grained Perception
Missing Tiles	63	40	Visual-Jigsaw (40)	12
Hidden Object	76	21	OmniSpatial (21)	12
Dot Shape	24	27	SPair-71k (27)	12
Color Swap	15	12	GQA (12)	12
C2: Spatial Reasoning

Appendix

Table 9: Task correspondence and demonstration-pool composition. Toy and Real report evaluation counts; source counts in parentheses sum to Real in each row. Pool reports demonstration counts. Target-side primary task labels define these associations, independently of ICL assignments; zero denotes an absent primary alignment.

Model	Toy	Real	Model	Toy	Real
GPT-5.6-Sol	73.90	59.4750	Gemma4-12B-it	20.60	30.3300
Gemini-3.1-Pro-Preview	59.00	61.2750	GLM-4.6V-Flash	17.20	34.7400
Claude-Opus-4.8-thinking	55.00	50.4500	Ovis2.5-9B	17.00	28.2300
Qwen3.8-27B	50.30	58.2000	InternVL3.5-8B	15.10	28.0250
Qwen3.5-27B	42.40	51.1500	Qwen3-VL-8B-Instruct	13.90	28.1250
Gemma4-31B-it	34.70	47.8700	MiMo-VL-7B-RL	14.40	24.2750

Appendix

Table 10: The 25 paired scores used for cross-setting correlation. Values are the stored percentage scores used by the plotting script. Each model appears once; the two blocks are a compact presentation of the same 25-observation analysis.

Real-world aggregate or subset	Samples	Pearson $r$	95% CI
All samples, sample mean	400	0.960	$[0.913,0.981]$
All sources, source mean	400	0.964	$[0.902,0.987]$
Without SpatiaLQA, sample mean	350	0.946	$[0.869,0.979]$
Without SpatiaLQA, capability mean	350	0.933	$[0.837,0.972]$
Single-choice only, sample mean	236	0.939	$[0.860,0.975]$
Non-single-choice, sample mean	164	0.857	$[0.719,0.929]$

Appendix

Table 11: Sensitivity to real-world aggregation and answer format. All rows use the same 22 open-weight models and fixed toy-game capability-balanced scores. Sample means weight retained samples equally; capability means weight the four capability means equally. Source means weight the 11 source means equally. Intervals use 10,000 paired model-bootstrap replicates.

Removed source	Samples left	Sample mean $r$	Capability mean $r$
All-Angles-Bench	380	0.961	0.963
CAPTURe-real	370	0.958	0.960
GQA	388	0.959	0.959
IKEA Manuals at Work	395	0.956	0.956
OmniSpatial	339	0.965	0.964
ReasonMap	386	0.959	0.958

Appendix

Table 12: Leave-one-source-out correlations. Each row removes one real-world source while retaining all 22 models and the same toy-game capability-balanced scores. The two right columns differ only in how the remaining real-world samples are aggregated. The retained predictions and scoring rules are unchanged, allowing comparison under the two real-world aggregation rules.

Panel of Figure 7 — Figure 7: Conversation structure for one-shot evaluation. Italic text denotes sample-specific content, not additional literal instructions. The demonstration’s final-answer instruction is reproduced verbatim; the target uses the unchanged benchmark prompt.

Model	Setting	C1	C2	C3	C4	Overall-400
Qwen3.5-2B	Zero-shot	41.00	23.00	16.51	47.00	31.88
	1-shot seed 11	36.00	20.00	22.22	34.00	28.05
	1-shot seed 23	38.00	23.00	22.08	41.00	31.02
	1-shot seed 37	39.00	20.00	22.22	35.00	29.06
	1-shot mean	37.67	21.00	22.17	36.67	29.38
Qwen3.5-4B	Zero-shot	59.00	38.00	28.87	54.00	44.97

Appendix

Table 13: Zero-shot and per-assignment one-shot scores. C1–C4 denote fine-grained perception, spatial reasoning, constraint reasoning, and sequential planning. Overall-400 averages the four equally sized target groups. Scores are percentages. Bold rows report the three-assignment means; the remaining rows show the zero-shot baseline or one-shot scores for seeds 11, 23, and 37.

Model	Mean $\Delta$	SD	Observed $\Delta$ range	Gains / seeds
Qwen3.5-2B	$-2.50$	1.51	$[-3.83,-0.86]$	0/3
Qwen3.5-4B	$-4.01$	2.10	$[-5.92,-1.76]$	0/3
Qwen3.5-9B	$+1.49$	2.26	$[-1.09,+3.14]$	2/3
Qwen3.8-27B	$-0.33$	0.92	$[-1.39,+0.31]$	2/3

Appendix

Table 14: Overall changes and assignment variability.

\Delta

is relative to the same model’s zero-shot score. SD is the sample standard deviation across the three one-shot scores. Changes and SD are in percentage points. Ranges are observed minima and maxima, not confidence intervals; the last column counts how many of the three assignments improve over the zero-shot baseline.

Model	Non-thinking	Main evaluation	Gain
Qwen3.5-2B	13.07	19.30	+6.23
Step3-VL-10B	18.25	24.20	+5.95
Gemma4-31B-it	27.41	38.50	+11.09
GPT-5.6-Sol	28.27	69.80	+41.53

Appendix

Table 15: Overall scores under different inference settings. Both columns use the full 1,400-sample benchmark. Main evaluation reports the scores from the main experiments; Gain is its difference from Non-thinking in percentage points.

VLM Evaluation

Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.

See all 998 papers

Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints
Jul 23, 2026Liqiang Jing, Xiong Zhou, Siddharth Varia +3VLM Evaluation Visual Reasoning
University of Texas at Dallas · Amazon Web Services
Vero: An Open RL Recipe for General Visual Reasoning
Apr 6, 2026Gabriel Sarch, Linrong Cai, Qunzhong Wang +3Reasoning for VLMs Visual Reasoning
Princeton University
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
May 25, 2026Longteng Guo, Yifan Wang, Pengkang Huo +4Visually Grounded Reasoning Visual Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences

Jul 23, 2026J/K move · Enter open · S save

Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

Liqiang Jing, Xiong Zhou, Siddharth Varia, Neha Anna John+2

University of Texas at Dallas · Amazon Web Services

VLM Evaluation Visual Reasoning VLM Reasoning RL for VLMs

Open paper AskPDF arXiv

While Large Vision-Language Models (LVLMs) exhibit strong perceptual capabilities, they remain vulnerable in visual reasoning tasks. Existing benchmarks largely focus on symbolic mathematical or scientific problems and simple vision-centric tasks, offering limited assessment of complex visual reasoning and logical consistency, a critical requirement for reliable reasoning systems. We introduce ConVBench, a complex vision-centric reasoning benchmark in which each image is paired with two logically equivalent questions across six categories: action and state, complex counting, spatial reasoning, causal and intent understanding, commonsense reasoning, and temporal perception. To complement this benchmark, we define two evaluation metrics, logical consistency and robust accuracy, that jointly assess both the correctness and consistency of model responses. We further present ConVLM, which improves LVLM reasoning through Group Relative Policy Optimization (GRPO)-based reinforcement learning with a novel consistency reward. This method leverages automatically generated logically equivalent question-answer pairs and a dual-reward design combining accuracy- and consistency-based signals, encouraging agreement between paired responses. The framework functions effectively with or without strict answer supervision.

Abstract

VLM Evaluation

Explore similar work

Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

Vero: An Open RL Recipe for General Visual Reasoning

Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning