Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs

Hengwei Ye; Yuanting Guan; Yuxuan Ge; Tianying Zhu; Zhenhan Guan; Yijia Zhong; Yijing Zhang; Han Zhang

cs.CLMar 2, 2026

Children's Intelligence Tests Pose Challenges for MLLMs? KidGym: A 2D Grid-Based Reasoning Benchmark for MLLMs

Authors: Hengwei Ye, Yuanting Guan, Yuxuan Ge, Tianying Zhu, Zhenhan Guan, Yijia Zhong, Yijing Zhang, Han Zhang,

Organizations: ShanghaiTech University

Abstract

Multimodal Large Language Models (MLLMs) combine the linguistic strengths of LLMs with the ability to process multimodal data, enbaling them to address a broader range of visual tasks. Because MLLMs aim at more general, human-like competence than language-only models, we take inspiration from the Wechsler Intelligence Scales - an established battery for evaluating children by decomposing intelligence into interpretable, testable abilities. We introduce KidGym, a comprehensive 2D grid-based benchmark for assessing five essential capabilities of MLLMs: Execution, Perception Reasoning, Learning, Memory and Planning. The benchmark comprises 12 unique tasks, each targeting at least one core capability, specifically designed to guage MLLMs' adaptability and developmental potential, mirroring the stages of children's cognitive growth. Additionally, our tasks encompass diverse scenarios and objects with randomly generated layouts, ensuring a more accurate and robust evluation of MLLM capabilities. KidGym is designed to be fully user-customizable and extensible, allowing researchers to create new evaluation scenarios and adjust difficuly levels to accommodate the rapidly growing MLLM community. Through the evaluation of state-of-the-art MLLMs using KidGym, we identified significant insights into model capabilities and revealed several limitations of current models. We release our benchmark at: https://bobo-ye.github.io/KidGym/.

Figures & tables

Figure 1: Previews of 12 tasks in KidGym . The circular chart in the upper-right corner of each subfigure represents the cognitive abilities required by the task: E for Execution , M for Memory , L for Learning , P for Planning , and PR for Perception Reasoning .

Benchmarks	Target	Difficulty Level	User Extensible	Capabilities	Dynamic/Static
Crafter	RL	✗	✗	/	Dynamic
MiniGrid	RL	✓	✓	/	Dynamic
LogicGame	LLM	✓	✗	Learning Planning Execution	Static
EgoPlan	MLLM	✗	✗	Planning	Dynamic
MileBench	MLLM	✗	✗	Memory	Static
Countgd	MLLM	✗	✗	Counting	Static

Table 1: Comparison of KidGym with existing benchmarks across target paradigm, difficulty-level support, user extensibility, evaluated capabilities, and dynamic vs. static settings.

Figure 2: A KidGym task frame comprises a scene map, a backpack, and a hint bar. We provide varied agent skins, backgrounds, and scene-specific items; backpack slots and in-scene items are letter/number-labeled for identification. Resolution and grid layout are specified in Appendix B.1 .

Methods	L	CL	SE	SO	MA	FI	PU	PL	CO	DMA	MMA	MDE	MFI
Closed-Source Models (API)
o3	1	1.00	1.00	0.97	0.87	0.83	0.26	1.00	0.30	0.90	0.44	1.00	0.81
	2	0.98	1.00	0.95	0.42	0.52	0.11	0.98	0.26	0.47	0.18	1.00	0.50
	3	0.92	1.00	0.97	0.27	0.30	0.06	0.71	0.13	0.17	0.05	1.00	0.37
GPT-5	1	1.00	1.00	1.00	0.97	0.74	0.30	1.00	0.36	0.95	0.62	1.00	0.77
	2	0.96	1.00	0.99	0.43	0.60	0.06	1.00	0.18	0.47	0.10	1.00	0.61

Table 2: Zero-shot performance comparison of MLLMs across 12 KidGym tasks. “L” denotes the task level. Performance is measured by the success rate over 100 rounds under the ground-truth optimal solution , rounded to two decimal places.

Figure 3: Five-dimensional capability radar chart. The chart on the left shows the capability scores of the closed-source models, while the chart on the right shows those of the open-source models.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

	$32\times 32$	$64\times 64$	$96\times 96$
o3	0.27	0.30	0.59
GPT-4o	0.00	0.00	0.00
Claude-3.7-Sonnet	0.14	0.54	0.79
DeepSeekVL-2	0.12	0.12	0.17
QwenVL-2.5(7B)	0.00	0.00	0.00
QwenVL-2.5(72B)	0.00	0.00	0.00

Appendix

Table 3: Resolution Experiment on Counting (CO) Task

	CL	SE	SO	MA	FI	PU
L1	0.98 (39/40)	1.00 (38/38)	1.00 (45/45)	1.00 (44/44)	1.00 (41/41)	1.00 (39/39)
L2	0.95 (37/39)	1.00 (41/41)	0.97 (36/37)	0.98 (39/40)	1.00 (38/38)	1.00 (43/43)
L3	0.93 (40/43)	1.00 (43/43)	0.95 (38/40)	0.97 (37/38)	1.00 (43/43)	1.00 (40/40)

Appendix

Table 4: Human performance results on tasks CL, SE, SO, MA, FI, PU.

	PL	CO	DMA	MMA	MDE	MFI
L1	1.00 (43/43)	1.00 (37/37)	1.00(41/41)	0.97 (37/38)	1.00 (39/39)	1.00 (41/41)
L2	1.00 (38/38)	1.00 (44/44)	1.00(40/40)	0.95 (37/39)	1.00 (41/41)	1.00 (43/43)
L3	0.95 (39/41)	1.00 (41/41)	1.00(41/41)	0.92 (42/45)	1.00 (42/42)	1.00 (38/38)

Appendix

Table 5: Human performance results on tasks PL, CO, DMA, MMA, MDE, MFI.

	Execution	Memory	Learning	Planning	Perception Reasoning
Closed-Source Models (API)
o3	95	67	80	30	43
GPT-5	95	67	98	30	46
GPT-4o	23	49	43	7	21
Gemini-2.5-pro	100	70	79	31	48
Gemini-2.5-flash	63	50	59	15	31

Appendix

Table 6: Comparison of MLLM’s capability scores.

Methods	L	CL	SE	SO	MA	FI	PU	PL	CO	DMA	MMA	MDE	MFI
Closed-Source Models (API)
o3	1	1.00	1.00	0.97	0.95	0.81	0.24	1.00	0.25	0.92	0.47	0.99	0.81
	2	0.98	1.00	0.97	0.70	0.63	0.12	0.98	0.25	0.51	0.50	1.00	0.52
	3	0.92	1.00	0.89	0.73	0.31	0.03	0.81	0.12	0.20	0.71	1.00	0.35
GPT-4o	1	0.95	1.00	0.83	0.80	0.58	0.25	0.95	0.05	0.68	0.16	0.97	0.64
	2	0.96	0.92	0.59	0.31	0.37	0.02	0.38	0.03	0.10	0.01	1.00	0.27

Appendix

Table 7: Comparison of MLLM performances across 12 KidGym tasks in CoT.

Methods	L	CL	SE	SO	MA	FI	PU	PL	CO	DMA	MMA	MDE	MFI
Closed-Source Models (API)
o3	1	1.00	0.96	0.83	0.89	0.70	0.25	1.00	0.41	0.96	0.66	1.00	0.73
	2	0.98	1.00	0.88	0.77	0.55	0.06	0.99	0.16	0.59	0.63	1.00	0.56
	3	0.99	0.99	0.74	0.79	0.27	0.03	0.86	0.15	0.18	0.54	0.99	0.26
GPT-4o	1	0.32	0.80	0.46	0.38	0.32	0.21	0.62	0.00	0.30	0.08	0.54	0.36
	2	0.29	0.47	0.18	0.22	0.15	0.08	0.13	0.00	0.14	0.00	0.88	0.09

Appendix

Table 8: Comparison of MLLM performances across 12 KidGym tasks in ICL.

Figure 4: An example of task Classification (Level1). In this example, the goal is: “ Place strawberry in red basket and orange in yellow basket respectively. ”.

Figure 5: An example of task Counting (Level1). In this example, the goal is: “ Collect 3 pizzas. ”.

Figure 6: An example of task Selection (Level1). In this example, the goal is: “ Remember the toy car in the first image and then select it from the scene. ”.

Figure 7: An example of task Memory Decode (Level1). In this example, the goal is: “ Remember and select the item corresponding to the snail in the scene based on the relationship. ”.

Figure 8: An example of task Puzzle (Level1). In this example, the goal is:“ Complete the block puzzle based on the target graphic. ”.

Figure 9: An example of task Filling (Level1). In this example, the goal is: “ Complete the animal puzzle based on the target graphic. ”.

Figure 10: An example of task Memory Filling (Level1). In this example, the goal is: “ Remember and complete the animal puzzle based on the target graphic. .”.

Figure 11: An example of task Maze (Level1). In this example, the goal is: “ Use the orange key to open the orange door and then obtain the diamond. ”.

Figure 12: An example of task Decode Maze (Level1). In this example, the goal is: “ Use the yellow key to unlock the red door, then obtain the diamond. ”.

Figure 13: An example of task Memory Maze (Level1). In this example, the goal is: “ Use the blue key to unlock the blue door, then obtain the diamond hidden in the treasure chest using your memory. ”.

Figure 14: An example of task Sorting (Level1). In this example, the rule is: “ The lighter the animal is, the faster it is. ” and the goal is: “ Rank the animal in the backpack from fast to slow by speed in position I, II ”.

Figure 15: An example of task Placement (Level1). In this example, the goal is: “ Place the chick in the opposite direction to the west of the dog. ”.

Figure 16: “ Example image for Classification Task ”

Figure 17: “ Example image for Counting Task ”

Figure 18: “ Example image for Selection Task ”

Figure 19: “ Example image for Memory Decode Task ”

Figure 20: “ Example image for Puzzle Task ”

Figure 21: “ Example image for Filling Task ”

Figure 22: “ Example image for Memory Filling Task ”

Figure 23: “ Example image for Maze Task ”

Figure 24: “ Example image for Decode Maze Task ”

Figure 25: “ Example image for Memory Maze Task ”

Figure 26: “ Example image for Sorting Task ”

Figure 27: “ Example image for Placement Task ”

Multimodal Model Evaluation

Multimodal model evaluation measures how reliably AI systems understand and generate content across text, images, audio, video, and other data.

See all 770 papers

GIM: Evaluating models via tasks that integrate multiple cognitive domains
May 18, 2026Rohit Patel, Alexandre Rezende, Steven McClainLLM Evaluation Test-Time Scaling
Meta Superintelligence Labs
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Visually Grounded Reasoning VLM Reasoning
University of Cambridge
Evaluating Cognitive Age Alignment in Interactive AI Agents
May 18, 2026Yifan Shen, Jiawen Zhang, Jian Xu +4LLM Agent Evaluation AI Agent Benchmarks
1PediaMed AI · University of Illinois Urbana-Champaign · 4Peking University +2

May 18, 2026J/K move · Enter open · S save

GIM: Evaluating models via tasks that integrate multiple cognitive domains

Rohit Patel, Alexandre Rezende, Steven McClain

Meta Superintelligence Labs

LLM Evaluation Test-Time Scaling

Open paper AskPDF arXiv

As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficulty: escalating knowledge demands (GPQA, HLE) or removing knowledge entirely in favor of abstract reasoning (ARC-AGI). The first conflates memorization with capability; the second divorces reasoning from the practical contexts in which it matters. We take a different approach. The Grounded Integration Measure (GIM) is a benchmark of 820 original problems (615 public, 205 private) where difficulty comes from integration; individual problems require coordinating multiple cognitive operations (constraint satisfaction, state tracking, epistemic vigilance, audience calibration) over broadly accessible knowledge, so that reasoning stays grounded in realistic tasks without being gated on specialized expertise. Each problem is an original expert-authored composition, majority with rubric-decomposed scoring (median 6 independently judged criteria). A balanced public--private split provides built-in contamination diagnostic. We calibrate a continuous response 2-parameter logistic (2PL) IRT model over >200k prompt-response pairs across 28 models, producing robust ability estimates that correctly order test-configurations even when raw accuracy is distorted by errors or missing data, addressing a common challenge in benchmark reporting. Using this framework, we present a comprehensive leaderboard spanning 22 models and 47 test-configurations (unique model, thinking-level pairs), and conduct what is to our knowledge the most extensive published study of how test-time compute trades off against model capability on a fixed benchmark: 11 models swept across 35 test-configurations. We observe that within-family configuration choices, such as thinking budget and quantization, matter as much as model selection. We release the evaluation framework, calibrated IRT parameters, and all public problems.

Abstract

Multimodal Model Evaluation

Explore similar work

GIM: Evaluating models via tasks that integrate multiple cognitive domains

ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models

Evaluating Cognitive Age Alignment in Interactive AI Agents