3DCodeVerse: Exploring 3D Worlds as Code

Yipeng Gao; Ziyao Zeng; Jinfa Huang; Yijiang Li; Haolin Xiong; Wang Qin; Xinlei Yu; Youheng Yao

cs.CVOct 15, 2026

3DCodeVerse: Exploring 3D Worlds as Code

Authors: Yipeng Gao, Ziyao Zeng, Jinfa Huang, Yijiang Li, Haolin Xiong, Wang Qin, Xinlei Yu, Youheng Yao,

Organizations: University of Southern California · Yale University · University of Rochester · University of California, San Diego ·

Abstract

We introduce 3DCodeVerse, a research ecosystem that enables coding agents to create editable and verifiable 3D content at scale. 3DCodeVerse comprises large-scale, high-quality datasets of executable 3D code, a coding harness for spatial reasoning and automated verification, and practical guidance for 3D coding. First, we construct 3DCodeVerse-2M, a large-scale curated dataset of standalone 3D executable code. We build this corpus by collecting open-source data, converting specialized representations, and generating new programs with coding agents. Each program is packaged with the source files and associated assets required for execution in its declared environment, along with available descriptions and verification records. Second, we develop the 3DCodeVerse Harness, a multi-agent directed acyclic graph framework for generating and refining 3D content. It uses execution checks, spatial measurements, and visual feedback to automatically evaluate and revise generated programs without human feedback during generation. Third, we conduct systematic empirical studies of 3D coding, establishing baselines with fine-tuned open models and deriving practical guidance through component-level analyses of our harness. Extensive experiments demonstrate that fine-tuning on our dataset significantly improves the 3D coding capabilities of open-source models. Notably, our 3DCodeVerse harness achieves a 90% win rate against Claude Design (3D Object) in pairwise human evaluations, when both are utilized by Claude Opus 5.5.

Figures & tables

	Dataset					Reported size		Harness
Work	Static	Artic.	Scene	Graphics	Code Types	Count	Unit	Iterative refinement	Visual feedback	Geometry checks
Datasets and benchmarks
Shaders21k ( Baradad et al., 2022 )	✗	✗	✗	✓	1	21 K	Programs	✗	✗	✗
BlendNet ( Du et al., 2024 )	✓	✗	✗	✗	1	8 K	Pairs	✗	✗	✗
CAD-Recode ( Rukhovich et al., 2025 )	✓	✗	✗	✗	1	1 M	Programs	✗	✗	✗
MeshCoder ( Dai et al., 2026 )	✓	✗	✗	✗	1	1 M	Pairs	✗	✗	✗

Table 1: Comparison of 3D coding datasets, benchmarks, and harnesses. Dataset columns show released content, and Graphics marks dedicated shader, material, lighting, or animation data. Sizes follow each source’s own unit and are not directly comparable. Harness columns summarize each work’s inference-time loop. ◐ denotes partial support and — means no new dataset.

Figure 2: Curation and composition of 3DCodeVerse -2M. Left: We collect programs from public datasets and generate more with our harness. Middle: Our 3D coding harness, equipped with modeling skills and inspection tools (e.g., 3D IoU, collision checks), builds each program, judges the output, and refines it until it passes. Each verified program is paired with rendered images and captions, plus meshes and videos when available. Right: The resulting corpus covers 3D objects, articulated objects, scenes, and graphics across multiple programming languages and engines.

Figure 3: Overview of the 3DCodeVerse Harness. Given an input prompt, an LLM planner defines the target asset, and a DAG coordinates sub-agents to code and assemble a 3D program (left). Execution checks, spatial measurements, and rubric-guided VLM judgments provide automated feedback (center) for targeted code revisions by a fix agent (bottom). The right panel illustrates successive refinement rounds, with the highest-scoring program selected as the final output.

Figure 4: Main results of 3DCodeVerse Harness across four 3D coding tasks. Panels show (a) 3D objects, (b) articulated objects, (c) scenes, and (d) graphics. Blue bars report Human Preference (%), and orange bars report Judge Score (0–100; failed generations receive a score of zero). Hatched bars denote each backbone’s native harness, while solid bars denote 3DCodeVerse Harness. The final group in (a) compares Claude Design with 3DCodeVerse Harness, both using Claude Opus 5.5.

Model	3DCodeBench	Blender	CadQuery	OpenSCAD	GLSL	Three.js
	(212)	(103)	(200)	(50)	(200)	(40)
Qwen3.5-9B	0.0	1.9	10.5	50.0	33.0	72.5
+ SFT	84.9	90.3	98.5	86.0	56.0	74.5
$\Delta$	+84.9	+88.4	+88.0	+36.0	+23.0	+2.0
Qwen3.8-27B	51.9	74.8	65.5	80.0	57.0	72.5
+ SFT + DPO	92.9	93.2	98.5	90.0	71.5	92.5

Table 2: Fine-tuning with 3DCodeVerse -2M makes open models executable. Execution rate (%) before and after LoRA fine-tuning (training-set size in parentheses);

\Delta

is the change and the better value of each pair is in bold. A program counts when it runs in the dialect’s runtime and produces a result (GLSL: the shader compiles). All models are decoded at

T{=}0.7

with one seed on the same fixed test sets.

Figure 5: Qualitative comparisons and automated refinement. (a) Outputs for identical prompts and backbones using the native harness ( Without ) versus 3DCodeVerse Harness ( With ). (b) Selected rounds of automated refinement for objects and graphics. Numbers report rubric scores on a $[0,1]$ scale and are comparable within each pair or refinement sequence.

Figure 7: Applications. Code renders from our harness guide video generation with Gemini Omni Flash 1.1 for robot manipulation scenes (top) and Seedance 2.5 for a GPT-6 Astra scene (bottom).

Appendix figures & tables24 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 8: Components and model roles. A planner, a judge, and a coding agent are configured modularly per run; the harness orchestrates stage progression, iterative refinement, and candidate selection while providing deterministic spatial tools, verification gates, multi-view renderers, and rubric aggregation. Agent programs execute across seven isolated language runtimes; dashed circles indicate pluggable model interfaces.

Figure 9: From structural plan to file scopes and coding sessions , illustrated on the recorded trajectory of § A.6 . The planner decomposes the prompt into named parts with metric bounding boxes, materials, attachment relations, and an acceptance checklist; each target maps to its owned source files, and refinement tasks sharing files are grouped into file-disjoint sessions. In Round 1, nine revision tasks partition into eight file-disjoint groups; in Round 2, four tasks form four groups.

Figure 10: Observations, deterministic gates, and authoring-frame feedback. A measured geometric defect is returned as a structured negative verdict rather than a protocol error (left); a single verification routine serves simultaneously as an interactive agent tool, an end-of-round gate, and a generator of prioritized revision tasks (middle). Right: baseline-round diagnostics from § A.6 , with corrective translation vectors projected directly into the agent’s native authoring frame.

Figure 11: Vision–language judgment: evidence payload, structured schema, and score synthesis. The judge inspects structured telemetry and labelled multi-view montages without access to source code or generator reasoning (left); its schema enforces explicit visual observation prior to criterion scoring (middle). Right: baseline-round score computation for § A.6 , combining criterion means, majority-voted defects, and deterministic gate caps.

Figure 12: Recorded trajectory for the prompt “a three-legged wooden stool” on the Three.js runtime under the economy profile. Each column shows one round: the rendered view inspected by the judge, the composite score with its penalties and caps, deterministic gate findings, and VLM defect votes. Closed-loop refinement resolves all baseline floating and interpenetrating parts in Round 1 (

0.410\to 0.844

) and refines geometric detailing in Round 2 (

11{,}052

triangles, reaching

0.933

under the strict metric-footprint cap), completing in

4.0

active minutes at $0.74.

Figure 13: Articulated mechanism verification , illustrated on a recorded run for “a laptop with a hinged lid”. Link geometry is authored in Blender and kinematic structure in URDF; the build verifies rest-pose frame agreement, and the joint sweep articulates the mechanism across its motion limits. The dynamic sweep automatically detects a subtle $2.5$ mm hinge–base collision at the upper opening limit that is invisible at rest, enforcing physical articulability via the rubric cap.

Figure 14: Scene and procedural graphics tracks. (a) Scene generation executes a staged asset-and-zone assembly pipeline before entering the closed-loop refinement cycle with scene-specific gates. (b) Graphics generation synthesizes fragment shaders or OpenGL programs whose temporal frames are sampled at fixed timestamps, verified by statistical frame gates, and evaluated as multi-frame montages.

Bundle	Guidance	Standing route	Gate-fired route
part-contact	contact, weld-overlap, penetration, and stray-island limits of the connectivity gate	multi-part objects	connectivity, joint sweep
bbox-contract	plan dimensions: tolerance $\max(1\,\mathrm{cm},10\%)$ , error past $3\times$	object tracks	contract
repeats-and-mirrors	instances, mirror and radial arrays, instance naming	instances or symmetry	instance box
blender-forms	form-to-Blender-API techniques; five common lint errors	Blender, URDF+Blender	part not imported
cadquery-forms	CadQuery forms; fillet, revolve, and boolean rules	CadQuery	—
threejs-forms	form word to Three.js geometry; no CSG	Three.js objects	—

Appendix

Table 3: The domain skill library , listing each bundle’s authoring guidance alongside its standing and gate-triggered routing conditions.

Subset	Dialect	Samples	Share	Code MB	Render entries	Origin terms
DeepCAD	DeepCAD $\rightarrow$ CadQuery	128,201	47.55%	136.7	769,107	MIT
Shadertoy	GLSL	119,961	44.50%	394.7	1,196,826 †	per shader, mostly CC BY-NC-SA
Articraft (CadQuery)	CadQuery	6,882	2.55%	120.0	3,300 ‡	CC BY 4.0
Bioinspired3D	Blender Python	6,343	2.35%	9.3	31,715	MIT
Articraft (URDF)	URDF	6,146	2.28%	103.4	2,846 ‡	CC BY 4.0
Thingiverse-OpenSCAD	OpenSCAD	1,193	0.44%	4.9	5,965	per item

Appendix

Table 4: The packaged catalogue , one row per collection and dialect at the packaging snapshot. Origin terms never replace the dataset-level CC-BY-NC-SA-4.0 terms; the stricter applicable terms control reuse. Render entries count the render paths recorded per sample, which include mesh exports next to images. † Estimated from a

10\%

stratified sample of archive manifests. ‡ Indexed render paths in the initial release manifest; full eight-view and pose renders reside in the chunk archives (§ B.2 ). § Packaged coding-agent subsets at the release snapshot (§ B.2 ).

Mix	Base	Pairs	Mtok	cap/sample	Composition	Wall
5.2 K Blender	9B	5,193	3.8	mixed	Bioinspired3D + non-test factories	30 m
25.6 K	9B	25,600	26	1	4 dialects, merged single-dialect sets	1 h46
102 K	9B	101,851	107	1	CadQ. 40 K, GLSL 40 K, OpenSCAD all, Blender 5 K + 15.7 K boot	3 h25
116 K (bal)	9B	116,000	110	1	as above, Blender share 15% $\to$ 34% by repetition	2 h26
270 K	9B	270,173	293	1	all CadQuery, all GLSL, Blender 5 K + 15.7 K boot, OpenSCAD $\times$ 4	4 h46
364 K	9B	364,000	304	3	all six sources, no boot	6 h43

Appendix

Table 5: Supervised fine-tuning mixtures. Composition, token volume, and wall-clock training duration on

4\times\text{H100}

GPUs across the SFT mixtures evaluated in § 5 and § C.4 ( boot denotes execution-verified bootstrap programs synthesized by 3DCodeVerse Harness; cap/sample denotes the number of caption views per program).

Figure 15: Training loss dynamics of the three Qwen3.8-27B data-scale runs. (a) Cross-entropy loss per optimizer step; faint curves show raw 5-step logs and solid curves show an exponential moving average (

\tau=50

steps). (b) Smoothed loss trajectories plotted against epoch fraction to align cosine learning-rate schedules, annotated with terminal smoothed losses. Each trajectory tracks in-distribution optimization across its respective representation mixture from

293

K to

1.99

M rows.

A vs. B	$n$	$S_{\mathrm{all}}^{A}$	$S_{\mathrm{all}}^{B}$	$\Delta S_{\mathrm{all}}$	95% CI	W/L/T
Flash + 3DCodeVerse vs. Flash baseline	60	86.6	61.3	$+25.3$	$[19.0,31.6]$	56/4/0
Flash + 3DCodeVerse vs. Claude Design	60	86.6	37.4	$+49.2$	$[43.6,54.9]$	59/1/0
Opus baseline vs. Claude Design	60	77.2	37.4	$+39.9$	$[34.5,45.2]$	59/1/0
Flash baseline vs. Claude Design	60	61.3	37.4	$+23.9$	$[17.3,30.5]$	51/9/0
A vs. B	$n$	$S_{\mathrm{vis}}^{A}$	$S_{\mathrm{vis}}^{B}$	$\Delta S_{\mathrm{vis}}$	95% CI	W/L/T ∗
Flash + 3DCodeVerse vs. Flash baseline	60	88.1	69.3	$+18.8$	$[13.2,24.5]$	54/6/0

Appendix

Table 6: Paired object score comparisons (

0

–

100

scale). Upper panel: composite Judge Score

S_{\mathrm{all}}

incorporating deterministic defect penalties and caps. Lower panel: uncapped, scale-free visual score

S_{\mathrm{vis}}

(Eq. 7 ). W/L/T tallies per-prompt wins, losses, and ties by score difference.

System	$n$	Exec. (%)	Input (k)	Cached (k)	Output (k)	Est. USD
Claude Design	60	100.0	–	–	–	–
Flash baseline	60	93.3	0.3	0.0	9.4	0.035
Opus baseline	60	100.0	3.0	0.5	28.5	0.353
Flash + 3DCodeVerse	60	100.0	20,816.0	19,551.8	234.9	3.312

Appendix

Table 7: Object execution reliability, token telemetry, and prompt-cache efficiency. Usage is averaged across all

60

prompts and excludes post-run evaluation calls. Over

93.9\%

of 3DCodeVerse Harness input tokens are served from low-cost prompt cache.

Track	Backbone	$n$	$\Delta S_{\mathrm{all}}$	95% CI	W/L/T	$\Delta S_{\mathrm{vis}}$
Graphics	Flash	11	$+27.2$	$[9.0,45.3]$	10/1/0	$+19.1$
Graphics	Sonnet	11	$-5.4$	$[-20.8,10.1]$	6/5/0	$-4.3$
Graphics	Luna	11	$+6.8$	$[-1.2,14.8]$	8/3/0	$+6.5$
Graphics	Astra	11	$+30.4$	$[8.6,52.2]$	10/1/0	$+28.2$
Scenes	Flash	10	$+19.4$	$[0.2,38.6]$	9/1/0	$+15.4$
Scenes	Sonnet	10	$+0.1$	$[-31.1,31.3]$	3/6/1	$+2.8$

Appendix

Table 8: Paired cross-track gains on 3D scenes and procedural graphics (

0

–

100

scale). Differences compare 3DCodeVerse against the native baseline of the identical backbone across paired prompts (

n

), reporting both composite Judge Score (

\Delta S_{\mathrm{all}}

) and uncapped visual score (

\Delta S_{\mathrm{vis}}

Repeatability metric	Value
$\sigma_{\mathrm{judge}}$ , Gemini 3.1 Pro ( $3\times$ )	0.030
$\sigma_{\mathrm{judge}}$ , Gemini 3.8 Flash ( $3\times$ )	0.083
Paired A/A runs $n$	6
Mean difference $\bar{\Delta}$	$+0.043$
Median difference	$-0.008$
Std. deviation $\sigma_{\Delta}$	0.202

Appendix

Table 9: Judge precision vs. structural exploration across repeated runs. Left: paired statistics across two independent runs under identical settings, juxtaposed with Gemini 3.1 Pro judge repeatability (

\sigma_{\mathrm{judge}}=0.030

). Right: an illustrative prompt in which two independent runs synthesize valid but topologically distinct decompositions (a single monolithic mesh vs. a

10

-part assembly).

		3DCodeBench (212)			Blender (103)		CadQuery (200)		OpenSCAD (50)		GLSL (200)	Three.js (40)
Model	Training data	Exec	F	DINO	Exec	F	Exec	F	Exec	F	Exec	Exec
Zero-shot
Qwen3.5-9B	—	0.0	.000	.000	1.9	.008	10.5	.030	50.0	.261	33.0	72.5
Qwen3.8-27B	—	51.9	.214	.211	74.8	.439	65.5	.159	80.0	.415	57.0	72.5
Qwen3.5-9B + LoRA SFT
	5.2 K, Blender	73.1	.257	.228	93.2	.694	31.5	.081	32.0	.099	23.0	47.5

Appendix

Table 10: All fine-tuning mixtures. Full results behind Table 2 , including every training mixture and per-dialect [email protected]. All models are decoded at

T{=}0.7

with one seed and scored on the same fixed test sets. Exec : programs that run in the dialect’s runtime (%); for GLSL, shaders that compile. F : F-score at

\tau{=}0.05

between the generated and reference meshes, with failed programs scored 0. DINO : DINOv2-L cosine similarity between four-view renders of the generated and reference meshes (chance 0.159). Best model result per column in bold.

		3DCodeBench exec		3DCodeBench geometry		Blender held-out
Base $\rightarrow$ +DPO	pairs	before	after	F ${}^{\text{ok}}$	CD $\downarrow$	F ${}^{\text{ok}}$	CD $\downarrow$
execution feedback
9B, 5.2 K base $\rightarrow$ round 1	2,590	75.0–82.1	93.9	0.384	0.201	—	—
$\rightarrow$ round 2	587	93.9	96.2	0.383	0.200	—	—
9B, 102 K base	1,052	84.9	87.3	0.375	0.202	0.783	0.077
9B, 270 K base	909	90.1	89.6	0.388	0.204	0.778	0.078

Appendix

Table 11: Direct preference optimization across verifiable feedback signals (greedy decoding). Execution feedback pairs an executing completion against a failing completion for the same prompt; geometry feedback pairs the highest- and [email protected] completions; and termination feedback pairs a non-terminating greedy rollout against the shortest valid completion. All runs in this table use greedy decoding (

T{=}0

); multi-seed sampling (

T{=}0.7

, pass@4) is reported in Table 12 .

best-of- $N$ ( $T{=}0.7$ , $N{=}4$ )	per-seed exec				pass@4
9B, 5.2 K base	76.4	66.5	64.6	75.0	91.0
+ exec-DPO	90.6	93.4	86.3	88.2	99.1
9B, 270 K + geo-DPO	88.2	83.5	82.1	86.8	98.1

Appendix

Table 12: Best-of-

N

sampling at

T{=}0.7

, four seeds. Execution-feedback DPO lifts both the mean seed and the union: pass@4 reaches 99.1%.

Model	Harness	Human Pref. (%) $\uparrow$	VLM (%) $\uparrow$	SigLIP-2 $\uparrow$	Judge Score $\uparrow$
Same Model with Different Harnesses
Gemini 3.8 Flash	Gemini CLI	13%	15%	0.1356	61.3
Gemini 3.8 Flash	3DCodeVerse	87%	85%	0.1468	86.6
Claude Opus 5.5	Claude Code	21%	19%	0.1437	77.2
Claude Opus 5.5	3DCodeVerse	79%	81%	0.1490	88.5
GPT-6 Astra	Codex	31%	28%	0.1456	76.4

Appendix

Table 13: 3D object generation. Numerical values of Figure 4 a–c. Comparisons at a fixed backbone (upper) and against Claude Design (lower), on identical prompts. SigLIP-2 reports cosine similarity; Judge Score reports overall rubric quality (0–100).

Model	Prompts	$n$	$\Delta$	95% CI	W/L/T
GPT-6 Luna	hard	10	$+29.1$	$[14.1,44.1]$	10/0/0
GPT-6 Terra	hard	11	$+19.1$	$[1.3,36.8]$	8/2/1
GPT-6 Sol	hard	11	$+15.5$	$[3.1,27.9]$	8/2/1
Gemini 3.8 Flash	hard	8	$+40.1$	$[20.9,59.2]$	7/0/1
Gemini 3.8 Flash	mixed	8	$+52.3$	$[34.5,70.1]$	8/0/0

Appendix

Table 14: 3DCodeVerse Harness vs. one-shot generation across models. Scores are assigned by a fixed judge with two samples per output. In both settings, the GPT-6 models use the Codex CLI backend and Gemini 3.8 Flash uses the Gemini CLI backend.

\Delta

is the paired score difference in points, and W/L/T counts per-prompt wins, losses, and ties by score difference. The generation settings differ from those of the object evaluation in Figure 4 a.

	Score		Execution		$ / run
Model	Harness	1-shot	Harness	1-shot	Harness	1-shot
GPT-6 Sol	0.758	0.603	100%	100%	6.72	0.29
GPT-6 Terra	0.646	0.441	100%	92%	3.99	0.05
GPT-6 Luna	0.601	0.289	100%	75%	1.05	0.01
Gemini 3.8 Flash	0.625	0.146	100%	38–57%	2.41	0.04
Claude (Claude Code default, 1-shot)	—	0.472	—	75%	—	1.17

Appendix

Table 15: Mean score, execution reliability, and cost per run on the hard-prompt suite, comparing 3DCodeVerse Harness against one-shot generation. Execution rate is the percentage of programs that build and export valid geometry. Lower panel: paired comparisons on eight matched hard prompts under the same fixed judge (score differences in points on a

0

–

100

scale), illustrating both tier-bridging (Flash + 3DCodeVerse Harness vs. GPT-6 Sol one-shot) and frontier scaling (GPT-6 Sol + 3DCodeVerse Harness).

Mechanism	Metric	$N$	Without	With	Effect
Plan resampling on reset	Invalid plans	560 calls	4.7%	0.7%	$-85\%$ ( $p{=}0.0067$ )
dangling-link trigger	Dangling refs.	416 calls	2.1%	0.0%	$-100\%$ ( $p{=}0.038$ )
Coupling schema (§ C.5 )	Valid URDF mimic	10 tasks	3/10	10/10	$+70$ pp
Verdict/transport split	Redundant retries	3,677 calls	26.6%	1.3%	$-95\%$
+ bounded payloads	Uncached tok./req.	5,194 reqs	38,521	13,480	$-65\%$
+ bounded payloads	Median cost/round	142 rounds	$1.572	$0.950	$-40\%$

Appendix

Table 16: Component-level systems ablations and large-scale telemetry analysis. Each infrastructure mechanism in 3DCodeVerse Harness targets a specific reliability, topological, or compute-efficiency bottleneck and is evaluated on its direct operational metric across large-scale invocation logs.

Figure 16: Articulated code simulation vs. UWM rollouts on the generated asset (Initial State 1). Rows correspond to contact push, hold, and retreat control sequences ( $16$ commands at $20$ Hz). Columns show the initial observation, MuJoCo simulation of the generated 3D code, its reconstruction through UWM’s frozen VAE, the static last-frame reference, and all three UWM rollout seeds. Enlarged $2\times$ crops (dashed boxes) demonstrate that executable 3D code preserves sharp, consistent door and handle geometry both before and after VAE reconstruction, whereas UWM rollouts exhibit seed-dependent contour drift.

Figure 17: Articulated code simulation vs. UWM rollouts on the generated asset (Initial State 2). Control sequences, column layout, and fixed $2\times$ crops match Figure 16 , displaying all nine UWM rollouts for Initial State 2.

Agentic Code Generation

Agentic code generation uses AI agents to plan, write, test, and revise executable programs with tools and feedback.

See all 121 papers

Apr 2, 2026

VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation

Evaluating code generation models for 3D spatial reasoning requires executing generated code in realistic environments and assessing outputs beyond surface-level correctness. We introduce a platform VoxelCode, for analyzing code generation capabilities for 3D understanding and environment creation. Our platform integrates natural language task specification, API-driven code execution in Unreal Engine, and a unified evaluation pipeline supporting both automated metrics and human assessment. To demonstrate its utility, we construct VoxelCodeBench, a benchmark of voxel manipulation tasks spanning three reasoning dimensions: symbolic interpretation, geometric construction, and artistic composition. Evaluating leading code generation models, we find that producing executable code is far easier than producing spatially correct outputs, with geometric construction and multi-object composition proving particularly challenging. By open-sourcing our platform and benchmark, we provide the community with extensible infrastructure for developing new 3D code generation benchmarks and probing spatial reasoning in future models.

Yan Zheng, Florian Bordes

FAIR at Meta · The University of Texas at Austin

Code Generation Benchmarks 3D Spatial Reasoning 3D Scene Generation

Ask PDF

May 31, 2026

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

Procedural 3D modeling through code is emerging as a versatile paradigm, offering deterministic, engine-ready, and precisely editable assets that neural 3D generators inherently lack. Authoring such procedural content, however, demands deep expertise in 3D software APIs, parametric design, and code-level geometric reasoning. In this paper, we propose 3DCodeBench, a systematic benchmark for evaluating vision-language model (VLM) agents for procedural 3D generation in 3D modeling software. Specifically, 3DCodeBench evaluates how effectively 12 advanced VLMs can serve as procedural 3D modelers by translating text and image references into procedural code for 3D modeling software. Recognizing that automated metrics may not fully capture the perceptual quality of 3D shapes, we build 3DCodeArena, a ranking platform based on pairwise human preferences over generated 3D outputs. From extensive evaluations and results, we observe that: (1) Failures mostly arise from API mismatches, while successful renders still suffer from disconnected or floating 3D geometric components. (2) Test-time scaling, such as higher thinking budgets and multi-turn refinement, improves performance overall. Our findings highlight a critical need for high-quality procedural coding data to advance commercial VLMs. Furthermore, effective procedural 3D modeling requires a robust execution environment that provides high-fidelity feedback for iterative refinement. We release 3DCodeBench, including the curated large-scale dataset of multimodal (text/image) prompts, procedural code, 3D object triplets, evaluation protocol, and the public 3DCodeArena platform as a foundational toolkit for exploring VLM-based procedural 3D modelers.

Yipeng Gao, Lei Shu, Genzhi Ye +5

Google DeepMind · University of Southern California · Google Research

VLM Evaluation 3D Asset Generation Procedural Content Generation Code Generation Benchmarks

Ask PDF

Sep 29, 2026

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes

Frontier coding agents can now write and execute code that authors 3D environments, but whether they reliably understand 3D structure and precisely control scene state remains unclear. The generated 3D scene is a persistent, executable artifact: a convincing render can hide incorrect spatial relations, intersecting objects, or unintended modifications. We introduce Code4Scene, a benchmark of 190 Unreal Engine cases built from human-assembled scenes that evaluates coding agents on two complementary settings under a shared execution interface. Construction tests scene-level spatial reasoning from open-ended language specifications, where many realizations are valid; editing tests precise control of scene state, where the agent must recover the target scene from reference images while preserving everything else. Rather than scoring code or rendered views, Code4Scene evaluates the generated engine-native scene for task fulfillment, artifact integrity, and static physical validity, with edits additionally compared against withheld ground truth. Across 14 coding-agent configurations on the 95-case public set, construction and editing performance are strongly correlated but not interchangeable (Spearman

ρ= 0.78

): Claude Fable 5.1 leads construction, Gemini 3.8 Flash leads editing, and GPT-6 Astra narrowly leads overall. Spatial Composition is the weakest construction category for every agent, while editing remains imprecise: the best Repair F1 is only 0.527, and 35.8% of edits that fully recover the target still introduce unintended changes elsewhere in the scene. These results expose a gap between plausible 3D generation and reliable spatial reasoning and state control.

Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen +6

UC San Diego · UC Berkeley

AI Coding Agents 3D Scene Editing 3D Spatial Reasoning AI Agent Benchmarks 3D Scene Generation

Ask PDF

Abstract

Agentic Code Generation

Explore similar work

VoxelCodeBench: Benchmarking 3D World Modeling Through Code Generation

3DCodeBench: Benchmarking Agentic Procedural 3D Modeling Via Code

Code4Scene: Benchmarking Coding Agents for Constructing and Editing 3D Scenes