Organizations: New York University · Carnegie Mellon University · UNC, Chapel Hill · University of Illinois Urbana-Champaign · The University of Texas at Austin · Google
Language models increasingly create 3D content through code, with practical workflows often relying on editing existing programs rather than regenerating them, for speed and precise control. Such edits require understanding what a program builds and foreseeing how a change reshapes it, yet prior benchmarks mostly evaluate generation or simple edits against a reference. In this paper, we introduce 3DCodeEdit, a benchmark of 2,592 instruction-guided editing tasks over 108 Blender programs that construct objects entirely in code. Its edits range from parameter to structural changes, style redesign, and scene composition, and are judged from renders against human-reviewed checklists. Across 17 models, the average success rate is only 37%, and even the strongest models complete fewer than 60% of structural and style edits. As models become more capable, their failures shift to geometric foresight, where the code runs but builds the wrong shape. We therefore study whether multi-turn feedback can compensate for this missing foresight. Feedback from a separate reviewer model that describes how the renders differ from the request helps most, more than maximum reasoning effort or an agent harness, and can let a 27B model surpass the single-shot rate of a 2.8T-parameter model.
Figures & tables
Task
Programs
Evaluation
Benchmark
Editing
End-to-End Code
Complex Code
Multiple Sources
Difficulty Levels
Reference-Free Eval.
CADBench ( Du et al., 2024 )
✗
✓
✗
✓
✗
✓
3DCodeBench ( Gao et al., 2026 )
✗
✓
✓
✗
✗
✗
3DHarnessBench ( Liu et al., 2026b )
✗
✓
✓
✗
✗
✗
Nova3D-Bench ( Noor et al., 2026 )
✗
✓
✓
✗
✓
✓
BlenderGym ( Gu et al., 2025 )
✓
✗
✗
✗
✗
✗
Table 1: Comparison of benchmarks for Blender code. 3DCodeEdit combines instruction-guided editing, end-to-end object construction, complex programs from multiple sources, graded edit difficulty, and reference-free evaluation. In end-to-end code , the program builds the entire object; complex code indicates an average program length exceeding 200 lines (Appendix B ); reference-free evaluation judges edits against the instruction rather than a reference output.
Figure 2: Construction of 3DCodeEdit. (1) A generator and an evaluator iteratively reconstruct OpenSCAD designs as editable Blender programs, preserving geometry while exposing parameters, parts, and assembly logic. We combine 53 reconstructed programs with 55 from 3DCodeBench. (2) For each program and edit type, we generate edit prompts paired with checklists of requested changes and preservation requirements. After filtering, selection, and ambiguity screening under human supervision, we obtain four tasks per program and edit type, totaling 2,592 tasks.
Type
Name
Operational definition
Geometric edits, graded by scope
G1
Parameter
Change one visible attribute (size, proportion, spacing, position, or orientation) of one part or part family.
G2
Component
Reconstruct the form of one part, together with a size change and one further visible change to it.
G3
Multi-component
Reconstruct and resize two separate parts independently.
G4
Structural
Reorganize how three or more regions meet, span, or support one another, changing the overall layout.
Open-ended edits
Table 2: Edit types in 3DCodeEdit and their operational definitions.
Model
Geometric
G1
G2
G3
G4
Open-ended
S
C
All
Exec.
Proprietary
Claude Opus 5.5
96.5
76.9
64.4
58.6
52.5
70.8
69.9
99.3
GPT-6.1-sol
95.6
71.3
65.3
59.3
57.6
65.7
69.1
96.1
Gemini 4 Argon
94.7
67.4
54.2
41.0
41.0
53.0
58.5
95.4
Gemini 3.8 Flash
94.7
68.8
62.7
53.7
34.5
36.3
58.4
92.0
Claude Sonnet 5.5
91.4
66.4
51.2
41.9
25.9
45.4
53.7
98.8
Claude Haiku 5.5
90.3
54.4
42.8
31.5
16.0
32.9
44.6
94.3
Table 3: Single-shot editing performance (%) on 3DCodeEdit. All-pass rate measures the percentage of edits whose programs execute, render, and satisfy every checklist item. G1–G4 denote geometric edits of increasing scope; S and C denote style and composition edits. Exec. reports execution and rendering success. Average is the mean over the 17 models. Five Qwen3.5 models are reported in Table 12 .
Figure 3: Statistics of 3DCodeEdit. (a) The 108 source programs by object category. (b) Program length in lines of code (log scale; shaded: middle 50%; ticks: individual programs). (c) Words per edit prompt by edit type (G1–G4, S, C; defined in Table 2 ; dot: median; bar: interquartile range; whisker: 5th–95th percentile), with the number of edit instructions below.
Figure 4: Progress in 3D program editing on the analysis subset (312 tasks). (a) All-pass rate of 44 models; lines join releases within four families, and the dashed line is the best rate so far. (b) Best rate per edit type among models released before April 2026 and among all models.
Share of failures
All-pass band
n
I
P
K
G
O
< 20%
19
11%
34%
37%
18%
0%
20–40%
19
11%
29%
23%
37%
0%
40–55%
23
10%
24%
22%
42%
2%
≥ 55%
15
7%
27%
23%
39%
4%
Table 4: Distribution of failure stages by editing performance. Each failure is assigned to the first stage at which the edit diverges from the instruction. I : instruction comprehension; P : program comprehension; K : code execution; G : geometric foresight; O : visibility in renders. As performance improves, failures shift from comprehension and coding toward geometric foresight.
Self-check
Indep. rev.
Editor
T0
Code
Render
P/F
Full
Self-rev.
GPT-6.1-sol
64.7
71.5
76.0
75.6
85.6
83.0
Claude Opus 5.5
67.6
68.3
74.0
81.7
83.0
83.3
Claude Sonnet 5.5
50.6
55.1
55.4
67.3
71.2
69.9
GPT-6-luna
36.9
42.6
49.7
59.0
65.7
61.5
Qwen3.8-27B
26.0
31.4
33.0
43.6
50.3
37.2
Table 5: All-pass rate (%) of multi-turn editing on the analysis subset (§ 4.4 ). T0 : single-shot edit. Indep. rev. : independent reviewer (GPT-6-luna, or GPT-6.1-sol for GPT-6-luna). Self-rev. : self-review. –: render self-check unavailable for text-only editors.
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Reconstruction of the NopSCADlib flat hinge, shown from one oblique view. The evaluator accepted round 22. Its first review began: “The main hinge leaf outline is substantially too regular and too large/deep compared with the source CSG leaf shape.”
Figure 6: Example source objects from the four source groups of 3DCodeEdit: a pineapple (546 lines of code) from 3DCodeBench, and a board-and-batten cottage (677), a small-parts cabinet (717), and a flat hinge (760) reconstructed from OpenSCAD, rendered from the source programs.
Figure 7: Twenty-four of the 108 source programs: 3DCodeBench objects (grey) and reconstructed designs (coloured).
Helical gear, G3, term
Before: enlarge each of the seven spiral-arranged openings by about 50%
After: enlarge each of the seven spiral-arranged openings to about 1.5 times its original diameter
Pulley scene, C, conflict between instructions
Before: add a pedestal-mounted two-link robotic arm with a large two-finger gripper reaching toward the pulley
After: add a two-link robotic arm rising from a pedestal block fixed to the gantry, with a large two-finger gripper reaching toward the pulley
Conflicting instruction: build a large open gantry spanning the workbench, carrying the robotic arm (after: the robotic arm’s pedestal block) above the turntable […]
Appendix
Table 6: Two instructions before and after ambiguity rewriting. Their checklist items were rewritten in the same way.
Benchmark
Programs
N
Mean
Med.
CADBench a
BlendNet
12,059
38
36
3DCodeBench
Reference
212
476
363
3DHarnessBench
Reference
100
876
784
Nova3D-Bench b
Generated
54
489
402
BlenderGym c
Start
280
51
40
BlenderBench
Start
27
25
24
Appendix
Table 7: Program length (lines) of Blender code benchmarks. Med.: median.
Figure 8: Two of the six verifier views for the task of Example C .
Figure 9: G1: rotate the fern’s outer fronds upward so they rise steeply instead of drooping.
Figure 11: G2: rebuild the dorsal fin as a smooth semicircular sail, twice as tall and longer.
Figure 13: G4: rebuild the desk legs as two A-frames with a thick shelf between them.
Figure 15: C: place the mushroom cluster on a field table under a canopy holding a lantern and a magnifying lens.
Renders
Code
Both
Verifier
Acc.
κ
Acc.
κ
Acc.
κ
Gemini 3.8 Flash (high)
90.4
0.80
83.5
0.62
91.7
0.82
GPT-6.1-sol
93.8
0.87
84.3
0.67
92.9
0.85
GPT-6-astra
91.9
0.83
84.3
0.67
91.9
0.83
Claude Opus 5.5
91.4
0.82
85.7
0.69
91.0
0.81
Claude Sonnet 5.5
85.2
0.69
81.0
0.57
84.3
0.67
Appendix
Table 8: Agreement with human labels on the 210 checklist items when the verifier sees the renders (our protocol), the original and edited programs, or both. Acc. in %. Gemini 3.8 Flash averages five calls on renders and three calls otherwise; the other verifiers use one call. GPT-6-astra uses reasoning effort medium here.
Verifier
Acc.
κ
Main verifier and reference
Gemini 3.8 Flash (low)
87.7
0.74
Gemini 3.8 Flash (medium)
90.0
0.79
Gemini 3.8 Flash (high) ∗
90.4
0.80
Gemini 3.8 Flash (max)
90.8
0.80
GPT-6.1-sol
93.8
0.87
Appendix
Table 9: Verifier agreement with human labels on 210 checklist items from the analysis subset. Acc. in %. Single call per item; Gemini 3.8 Flash rows average five independent calls. ∗ Configuration used to score all experiments.
Self-check
Indep. rev.
Editor
Code
Render
P/F
Full
Self-rev.
GPT-6.1-sol
0.3
0.3
3.4
2.8
1.5
Claude Opus 5.5
0.2
0.5
3.5
2.7
1.6
Claude Sonnet 5.5
0.4
0.6
4.3
3.8
4.7
GPT-6-luna
1.8
2.7
4.5
3.2
4.2
Qwen3.8-27B
1.5
1.6
5.5
5.0
2.6
Appendix
Table 10: Mean rounds of revision in each multi-turn setting of Table 5 (analysis subset, at most 8). –: text-only editor.
Setting
Luna
Sol
Single-shot edit
36.9
64.7
Single-shot edit, best effort
54.2
74.7
Codex harness
34.6
74.7
Codex harness + self-review
59.6
84.9
Multi-turn editing (self-review)
61.5
83.0
Appendix
Table 11: All-pass rate (%) of a coding agent (Codex CLI) and multi-turn editing on the analysis subset. Luna: GPT-6-luna; Sol: GPT-6.1-sol. Best effort: max for Luna, xhigh for Sol.
All-pass rate
Model
G1
G2
G3
G4
S
C
All
Exec.
Claude Opus 5.5
96.5
76.9
64.4
58.6
52.5
70.8
69.9
99.3
GPT-6.1-sol
95.6
71.3
65.3
59.3
57.6
65.7
69.1
96.1
Gemini 4 Argon
94.7
67.4
54.2
41.0
41.0
53.0
58.5
95.4
Gemini 3.8 Flash
94.7
68.8
62.7
53.7
34.5
36.3
58.4
92.0
Claude Sonnet 5.5
91.4
66.4
51.2
41.9
25.9
45.4
53.7
98.8
Appendix
Table 12: All-pass rate (%) of all 22 evaluated models by edit type (G1–G4: geometric edits graded by scope; S: style; C: composition), sorted by overall rate; the best value in each column is in bold. Exec.: percentage of edited programs that execute and render. Average: mean over the 22 models.
Model
Effort
G1
G2
G3
G4
S
C
All
Exec.
Tokens
Claude Opus 5.5
low
94.2
61.5
48.1
51.9
50.0
53.8
59.9
100.0
11.1
medium
96.2
73.1
55.8
51.9
51.9
76.9
67.6
99.4
14.7
high
98.1
71.2
61.5
57.7
51.9
73.1
68.9
99.4
17.9
xhigh
94.2
71.2
69.2
57.7
61.5
78.8
72.1
99.7
38.8
GPT-6.1-sol
low
94.2
73.1
59.6
59.6
51.9
63.5
67.0
96.2
7.1
medium
96.2
63.5
53.8
51.9
48.1
75.0
64.7
94.6
7.8
Appendix
Table 13: All-pass rate (%) by reasoning effort on the analysis subset (312 tasks; 52 per edit type). For the six models that are also in Table 12 , the medium level is their main run (at most 64k output tokens). All other levels use the provider’s maximum output length: 128k tokens for Claude and Kimi K3, 65,536 for Gemini 3.8 Flash, and no limit for GPT models. At this limit, the Claude models at max effort are truncated on up to 54% of a 24-task pilot, so we omit that level. Exec.: percentage of edited programs that execute and render. Tokens: mean output tokens per task in thousands, including reasoning tokens. The best overall rate of each model is in bold.
Model
Released
Family
G1
G2
G3
G4
S
C
All
Exec.
Qwen2.5-Coder-32B
2024-11-12
Qwen 27–32B
50.0
3.8
0.0
0.0
0.0
0.0
9.0
57.7
QwQ-32B
2025-03-06
Qwen 27–32B
46.2
5.8
0.0
1.9
0.0
0.0
9.0
41.3
Qwen3-32B
2025-04-29
Qwen 27–32B
48.1
9.6
0.0
0.0
0.0
0.0
9.6
52.2
Claude Sonnet 4
2025-05-22
Claude Sonnet
71.2
21.2
7.7
3.8
0.0
0.0
17.3
89.4
Gemini 2.5 Flash
2025-06-17
Gemini Flash
67.3
13.5
7.7
5.8
0.0
0.0
15.7
56.7
gpt-oss-120b ∗
2025-08-05
gpt-oss
75.0
9.6
0.0
1.9
0.0
0.0
14.4
75.3
Appendix
Table 14: The 44 models of the timeline analysis on the analysis subset (312 tasks), sorted by release date: all-pass rate (%) by edit type and overall, and the percentage of edited programs that execute and render (Exec.). ∗ : one of the 22 models of Table 12 . † : approximate release date. Claude Sonnet 4.6, Claude Sonnet 5, and GPT-5.6-luna are the medium levels of their effort sweeps.
Type
Opus 5.5
GPT-6.1-sol
Either
ϕ
G1
96.5
95.6
98.1
0.45
G2
76.9
71.3
85.6
0.40
G3
64.4
65.3
78.0
0.42
G4
58.6
59.3
73.6
0.39
S
52.5
57.6
74.3
0.23
C
70.8
65.7
86.1
0.18
Appendix
Table 15: The two strongest models on the full benchmark: all-pass rate (%) of Claude Opus 5.5 and GPT-6.1-sol, the percentage of tasks solved by at least one of them (Either), and the task-level correlation of their passes ( ϕ ).
Code
Definition
Share
Instruction: what to build
I-OMIT
A requested change is not attempted
4.1%
I-MISREAD
A different part, feature, or relation is built
5.3%
I-LITERAL
The wording is met but the intent is defeated
0.3%
I-OVERREACH
Rebuilt far beyond the request
1.0%
Program: understanding the source
Appendix
Table 16: Codebook of the error census. Each failure receives one stage, the earliest at which the edit diverges from the instruction, and one mechanism within it. Share: percentage of all failures of the 76 configurations on the analysis subset that receive the code.
Configuration
Pass
I
P
K
G
O
Crash
GPT-6.1-sol (xhigh)
74.7%
7%
45%
19%
21%
7%
17
Claude Sonnet 5.5 (xhigh)
73.7%
13%
21%
21%
42%
3%
3
Claude Opus 5.5 (xhigh)
72.1%
0%
27%
17%
53%
3%
0
GPT-6.1-sol (high)
70.8%
6%
32%
28%
30%
4%
20
GPT-6.1-sol (max)
70.5%
4%
29%
29%
29%
10%
21
Claude Opus 5.5 (high)
68.9%
10%
18%
16%
49%
8%
2
Appendix
Table 17: All-pass rate (Pass) and stage mix of failures for all 76 configurations on the analysis subset, sorted by all-pass rate. Stages: I instruction, P program, K code, G geometric foresight, O visibility. Crash: number of tasks, out of 312, in which the edited program crashes. Effort levels other than medium are given in parentheses.
We introduce neuralCAD-Edit, the first benchmark for editing 3D CAD models collected from expert CAD engineers. Instead of text conditioning as in prior works, we collect realistic CAD editing requests by capturing videos of professional designers, interacting directly with CAD models in CAD software, while talking, pointing and drawing. We recruited ten consenting designers to contribute to this contained study. We benchmark leading foundation models against human CAD experts carrying out edits, and find a large performance gap in both automatic metrics and human evaluations. Even the best foundation model (GPT 5.2) scores 53% lower (absolute) than CAD experts in human acceptance trials, demonstrating the challenge of neuralCAD-Edit. We hope neuralCAD-Edit will provide a solid foundation against which 3D CAD editing approaches and foundation models can be developed. Code/data: https://autodeskailab.github.io/neuralCAD-Edit
Text-guided 3D editing aims to modify existing 3D assets using natural-language instructions. Current methods struggle to jointly understand complex prompts, automatically localize edits in 3D, and preserve unedited content. We introduce Vinedresser3D, an agentic framework for high-quality text-guided 3D editing that operates directly in the latent space of a native 3D generative model. Given a 3D asset and an editing prompt, Vinedresser3D uses a multimodal large language model to infer rich descriptions of the original asset, identify the edit region and edit type (addition, modification, deletion), and generate decomposed structural and appearance-level text guidance. The agent then selects an informative view and applies an image editing model to obtain visual guidance. Finally, an inversion-based rectified-flow inpainting pipeline with an interleaved sampling module performs editing in the 3D latent space, enforcing prompt alignment while maintaining 3D coherence and unedited regions. Experiments on diverse 3D edits demonstrate that Vinedresser3D outperforms prior baselines in both automatic metrics and human preference studies, while enabling precise, coherent, and mask-free 3D editing.
Yankuan Chi, Xiang Li, Zixuan Huang +1
The Hong Kong University of Science and Technology · University of Illinois Urbana-Champaign
3D editing methods are usually tested on a single edit, yet an asset is built through a long sequence of revisions, each of which must implement the requested change while leaving everything else unchanged. We introduce EditHero, to our knowledge the first benchmark for long-horizon, part-level 3D editing, with natural-language instructions and target images for both geometry and texture. A deterministic assembly engine produces the exact target after every edit, and every sequence is reviewed by hand. We use EditHero to compare 2 opposite approaches to 3D editing. Non-agentic methods operate top down, regenerating the object from a learned 3D representation and inferring what to keep. In contrast, LLM/VLM agents operate bottom up, editing through code that inspects the mesh and rewrites only the parts required by instructions. The non-agentic methods often miss the requested change and disturb regions that should stay fixed. Most LLMs follow instructions more closely, and all of them preserve the unedited parts better, but each of their edits takes minutes. We will release the engine and the edit sequences to support research on reliable iterative 3D editing.
Ruihan Yu, Yu-Ju Tsai, Muyao Niu +9
Alaya Lab · The University of Tokyo · Institute of Science Tokyo +1