Coding agents combine visual understanding with program execution, offering a new route to automated 3D creation. By translating images and instructions into editable programs, they bring reconstruction, generation, and editing into a shared workflow. Yet existing benchmarks fragment evaluation across task scopes and protocols, leaving their capabilities across construction levels unclear. We propose Frontier3DBench, a unified benchmark of 145 tasks across 9 categories spanning object, scene, and interactive open-world creation, with joint evaluation of appearance, structure, and behavior. Experiments with 11 model-harness configurations reveal strengths that vary by level: GPT-6.1-sol leads the object and scene levels and achieves the highest overall score of 66.02/100, whereas GPT-6-astra leads interactive world construction. However, all configurations score lower on reference reconstruction than on text-driven generation in object and scene tasks. Producing plausible content thus remains easier than reproducing specified geometry, layouts, and visual details. Moreover, generated worlds average 83.08/100 on local mechanisms but only 20.82 on motion clearance and 30.26 on stateful workflows: individual mechanisms often function, whereas complete activities that require coordinated actions and persistent state remain difficult. These results identify reference fidelity and coordinated activity completion as central bottlenecks for coding agents in 3D creation.
Figures & tables
Figure 1: Frontier3DBench: 145 tasks in 9 categories spanning objects (L1), scenes (L2), and interactive open worlds (L3). Right: leaderboard of the 11 model–harness configurations (0–100, ↑ ).
Figure 2: Task formulation. Agents receive instructions and optional references or initial assets, then produce code and assets for Blender objects and scenes (L1/L2) or Unreal Engine open worlds (L3).
Task family
Tasks
Given input
Requirement
L1 · Objects
65
Output: editable Blender assets
Text2Code
15
Text instruction
Model the specified geometry and parts.
Img2Code
25
Multi-view images + instruction
Reconstruct an object across views.
Img2Code-SingleView
25
Single image + instruction
Reconstruct an object from one view.
L2 · Scenes
65
Output: editable Blender scenes
Brief2Scene
20
Text instruction
Compose objects, layout, and lighting.
Table 1: Frontier3DBench task families, counts, inputs, and requirements.
Level
Component
What it measures
L1/L2
V : visual fidelity
Fidelity to the instruction or reference and aesthetic quality, judged from rendered views using normalized 0–4 grades.
R : requirements
Satisfaction of task-specific dimensions, part counts, geometric features, and spatial relations, checked in native scenes or, for Brief2Scene, with an evidence checklist.
D : detail fidelity
Coverage and implementation accuracy of required details; the weakest measured execution or export variant determines the score.
I : image similarity
Agreement with the input reference in image structure, edges, and color or tone over the prescribed image regions.
L : object localization
Projected object placement, size, and count, using matched reference and candidate boxes with penalties for missing or extra instances.
T : target modification
Whether the requested edit is achieved in the designated region, within the permitted geometry, material, or lighting changes.
Table 2: Evaluation components and their measurement scope. Numerical weights are given in Appendix A .
Figure 3: Task-category profiles in overall-score order. Solid contours show each model’s task means; dashed contours show the mean over all 11 configurations.
Figure 4: Capability profiles for visual quality, structural and spatial consistency, and task fulfillment (0–100). Markers show dimension means over 65 L1, 65 L2, and 15 L3 tasks; outlines and labels mark level-wise maxima. Models follow overall-score order.
Figure 5: Overall score versus estimated average API cost per task across 145 tasks (log scale). Markers and the Pareto frontier use cost-range midpoints. Vertical bars show 95% task-bootstrap intervals for the overall score.
Figure 6: Object, scene, and interactive-world examples. Parentheses show task scores (0–100). World views are rendered from model-specific cameras. Minecraft reference: Akirby80.
Family
Items
Judgments
Mean τb
Top-one
Exact order
Img2Code
14
112
0.753
77.7%
43.8%
Img2Code-SingleView
16
128
0.659
78.1%
39.1%
Ref2Scene
8
64
0.781
84.4%
50.0%
Ref2Scene-RealPhoto
7
56
0.768
78.6%
53.6%
All families
45
360
0.727
79.2%
44.7%
Table 4: Agreement between human rankings and automatic task scores on four reference-conditioned task families (eight raters, 45 items, 360 rankings).
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Additional visual comparisons across objects, scenes, and interactive worlds. Columns group cases; rows show references and model outputs. (a) Four object and three scene cases compare runs of GPT-6.1-sol, DeepSeek-v4.1-Flash, Kimi-K3.1-Pro, and MiniMax-M3.1-Flash. (b) Three world cases each show two submitted project previews from GPT-6-astra, GPT-6.1-sol, and GPT-6-luna. Cases are drawn from the 145-task suite. World views use model-specific cameras; Valentine uses the same reference for both views. World reference credits: FromSoftware/Activision, Rockstar Games, and Poly Haven (CC0).