VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation

Yan Zheng; Florian Bordes

cs.LGApr 2, 2026

VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation

Authors: Yan Zheng, Florian Bordes

Organizations: FAIR at Meta · The University of Texas at Austin

Abstract

Evaluating code generation models for 3D spatial reasoning requires executing generated code in realistic environments and assessing outputs beyond surface-level correctness. We introduce a platform VoxelCode, for analyzing code generation capabilities for 3D understanding and environment creation. Our platform integrates natural language task specification, API-driven code execution in Unreal Engine, and a unified evaluation pipeline supporting both automated metrics and human assessment. To demonstrate its utility, we construct VoxelCodeBench, a benchmark of voxel manipulation tasks spanning three reasoning dimensions: symbolic interpretation, geometric construction, and artistic composition. Evaluating leading code generation models, we find that producing executable code is far easier than producing spatially correct outputs, with geometric construction and multi-object composition proving particularly challenging. By open-sourcing our platform and benchmark, we provide the community with extensible infrastructure for developing new 3D code generation benchmarks and probing spatial reasoning in future models.

Figures & tables

Figure 1: VoxelCodeBench evaluation pipeline. Given API documentation and a natural language task specification (left), models generate Python code manipulating voxels in 3D space (center). The generated code is executed within VoxelCode, an Unreal Engine based rendering server equipped with the Voxel Plugin 2.0 ( Careil, 2025 ) , producing visual outputs that can be evaluated by human annotators (right). Our benchmark spans three complexity tiers—from basic geometric primitives to compositional artistic scenes—enabling systematic assessment of spatial reasoning through code generation.

Figure 2: Example outputs using VoxelCode. Representative 3D voxel constructions generated by models across our benchmark, including characters and symbols, geometric and mathematical shapes, artistic animals, objects, vehicles, and natural objects, ranging from simple geometric primitives to complex multi-component objects.

Figure 3: Human annotation interface. Annotators view the prompt and rendered outputs from multiple viewpoints, then rate five dimensions: Has Object, Position Correct, Material Correct, Shape Correct, and Visual Quality (0–10).

Model	HasObj	Pos	Mat	Shape	Look
GPT-5	88.6	88.6	87.9	87.9	5.71
GPT-5 Mini	85.8	85.8	82.4	80.4	4.86
Claude Sonnet 4.5	86.7	86.7	84.6	80.4	5.01
GPT-5 Chat	77.9	77.9	75.9	69.7	3.66
Claude Opus 4	74.1	72.8	72.1	69.4	4.13
Claude 3.5 Sonnet	71.0	70.3	70.3	66.9	3.30

Table 1: Main results on VoxelCodeBench. Performance metrics (%) across models. HasObj : output contains visible object. Pos/Mat/Shape : position, material, shape correctness. Look : average visual quality score (0–10). Best results in bold .

	Symbolic		Geometric		Artistic
Model	Shape	Look	Shape	Look	Shape	Look
GPT-5	87.5	6.81	66.7	6.56	97.5	4.90
Claude Sonnet 4.5	90.3	6.77	52.8	5.28	89.5	4.16
Claude Opus 4	75.0	6.25	40.0	3.83	80.0	3.41
Gemini Pro	15.2	1.55	23.7	2.08	19.2	0.92

Table 2: Performance by task category. Shape correctness (%) and visual quality (Look, 0–10) across three reasoning dimensions. Best per category in bold .

Figure 4: Qualitative comparison across models, categories, and difficulty levels. Rows correspond to eight evaluated models; columns span three task categories (Symbolic, Geometric, Artistic) at Easy, Medium, and Hard difficulty. Empty cells indicate cases where code execution failed, resulting in no visible output due to errors in the model-generated code.

Model	Baseline	More Ex.	$\Delta$
GPT-5	97.5	96.2	$-$ 1.4
GPT-5 Mini	92.5	96.2	+3.7
GPT-5 Chat	94.9	61.5	$-$ 33.4
Claude Sonnet 4.5	89.5	72.0	$-$ 17.5
Claude 3.5 Sonnet	84.8	50.0	$-$ 34.8
Gemini Pro	19.2	44.0	+24.8

Table 3: Ablation: Effect of additional working examples. Shape correctness (%) on artistic composition tasks comparing baseline (2–3 examples) vs. extended prompts (5+ examples).

\Delta

indicates percentage point change.

Error Type	GPT-5	Claude Son. 4.5	Gemini Pro	Total
Invalid API Attribute	6	8	15	89
Execution Timeout/Crash	2	10	1	55
Type Unpacking Error	0	0	12	15
Syntax Error	0	0	14	15
Error Rate	0.8%	1.6%	5.6%	–

Table 4: Error type distribution. Common errors encountered during code execution. Error rate shows percentage of submissions with errors for each model.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Structured geometric detail generation. Code-based generation produces objects with consistent internal structures (interior views) and fine-grained details (ladders, weapons, floor layouts) that are difficult to achieve with surface-based neural 3D generation methods.

Code Generation Benchmarks

Code generation benchmarks measure how well AI models produce code across programming tasks, languages, and real-world settings.

See all 151 papers

3DCodeVerse: Exploring 3D Worlds as Code
Oct 15, 2026Yipeng Gao, Ziyao Zeng, Jinfa Huang +16Agentic Code Generation 3D Asset Generation
University of Southern California · Yale University · University of Rochester +6
Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes
Sep 29, 2026Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen +6AI Coding Agents 3D Scene Editing
UC San Diego · UC Berkeley
VoxProg: 3D Spatial Reasoning through Voxel Programming
Oct 21, 2026Zhichao Zhai, Menglei Zhao, Guikun Chen +43D Spatial Reasoning 3D Scene Layout Generation
Zhejiang University · vivo Mobile Communication Co., Ltd., Shenzhen, China

Oct 15, 2026J/K move · Enter open · S save

3DCodeVerse: Exploring 3D Worlds as Code

Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li+15

University of Southern California · Yale University · University of Rochester+6

Agentic Code Generation 3D Asset Generation Execution-Guided Code Generation Multi-Agent Coding

Open paper AskPDF arXiv

We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First, we construct 3DCodeVerse-2M, a large-scale curated dataset of standalone 3D executable code. We build this corpus by collecting open-source data, converting specialized representations, and generating new programs with coding agents. Each program is packaged with the source files and associated assets required for execution in its declared environment, along with available descriptions and verification records. Second, we develop the 3DCodeVerse Harness, a multi-agent directed acyclic graph framework for generating and refining 3D content. It uses execution checks, spatial measurements, and visual feedback to automatically evaluate and revise generated programs without human feedback during generation. Third, we conduct systematic empirical studies of 3D coding, establishing baselines with fine-tuned open models and deriving practical guidance through component-level analyses of our harness. Extensive experiments demonstrate that fine-tuning on our dataset significantly improves the 3D coding capabilities of open-source models. Notably, our 3DCodeVerse harness achieves a 90% win rate against Claude Design (3D Object) in pairwise human evaluations, when both are utilized by Claude Opus 5.5.

Abstract

Code Generation Benchmarks

Explore similar work

3DCodeVerse: Exploring 3D Worlds as Code

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

VoxProg: 3D Spatial Reasoning through Voxel Programming