Multimodal Large Language Models (MLLMs) have shown promise as interactive agents, yet precise, programmatic 3D graphics editing remains difficult to train because existing systems largely rely on closed-source model APIs and lack deterministic, state-verifiable execution environments. We present BlenderFORGE, a trainable data-environment-optimization framework for open-weight MLLM agents that edit explicit Blender scenes through Blender Python (bpy) code generation. BlenderFORGE targets four fundamental state-verifiable editing task families: object placement, shape-key editing, lighting adjustment, and material modification. It integrates three components: (1) a scalable perturbation-based data generation pipeline built from curated assets and large-scale 3D scene repositories; (2) an isolated Blender Sandbox for deterministic code execution, structured state export, and closed-loop reward computation; and (3) a two-stage training pipeline that uses offline teacher trajectories for cold-start Supervised Fine-Tuning (SFT) and sandbox-computed rewards for multi-turn tool-agentic Reinforcement Learning (AgentLoop-RL). Experiments across the four task families show that the resulting open-weight Qwen3-VL-8B and Qwen3.5-9B BlenderFORGE agents substantially improve over their base models and achieve competitive performance with several zero-shot proprietary MLLM baselines on selected metrics. Additional cross-benchmark evaluations and camera-reframing experiments provide evidence of transfer beyond the original training setting.
Figures & tables
Figure 1: Overview of the learning process under the BlenderFORGE framework. MLLMs learn the correspondence between visual changes and 3D-graphics edits from scene perturbations.
Figure 2: The BlenderFORGE architecture . BlenderFORGE is a unified framework that integrates a scalable 3D asset-based data generation pipeline with an interactive Blender sandbox to enable training MLLM agents in 3D graphics editing. (Left) Data Generation Pipeline: 3D assets are perturbed and filtered through a visibility check to ensure that the changes are observable. (Right) Training Loop: The validated data flows into training with an interactive Blender sandbox, enabling SFT and RL for MLLM agents.
Figure 3: Perturbation-based data generation pipeline. We illustrate the pipeline using object placement as an example. Starting from a base Blender scene, the pipeline removes redundant scene elements, selects editable objects or attributes, applies rule-based perturbations, and places aligned cameras so that the start–goal change remains observable. The resulting start and goal renderings are passed through a visibility check before the final (ss,sg) task pair is exported.
Table 1: Comparison of models on the test split of BlenderFORGE (metrics in Sec. 3.4.2 ). For Placement and Shape , SR@ τ (%) denotes success rate at threshold τ (higher is better); Med. denotes the median discrepancy (lower is better). PL ( ×10−3 ) and N-CLIP ( ×10−3 ) measure perceptual quality (lower is better). The w/o change row reports the discrepancy between the start scene ss and the goal scene sg . Within the Proprietary Models block and the combined Open-weight Models / Fine-tuned blocks, the best performance is marked with bold , and the second-best distinct score is underlined . The last two rows show gains of each BlenderFORGE model over its corresponding base: increases in SR (percentage points) and decreases in distance metrics. The 8B base Placement results use the N=200 evaluation in Tab. 2 .
Figure 4: Qualitative results of Qwen3-VL-8B-BlenderFORGE . Green and blue traces indicate positive progress, while orange and red traces indicate negative progress. Overall, the model improves the scene through fine-grained edits, although some detailed errors remain.
Table 2: Ablation study on the Placement task with N=200 for every row. SR@ τ (%) denotes success rate at threshold τ (higher is better); Med. denotes the median P-Chamfer distance (lower is better). PL ( ×10−3 ) and N-CLIP ( ×10−3 ) measure perceptual quality (lower is better). Avg. turns denotes the average number of interaction turns. Green deltas indicate improvement over the base model.
Table 3: Qwen3.5-9B transfer to BlenderGym. PL and N-CLIP use the ×10−3 convention; SR values are percentages. Dashes: SR and Med. are defined only for Placement and Shape-Key.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Client-server architecture of the Blender Sandbox. The outer Sandbox Manager orchestrates isolated Blender instances, each hosting a BlenderHTTP addon. Downstream MLLM agents interact exclusively through the proxy API.
Parameter
Value
Render engine
Cycles (GPU)
Samples / Max bounces
128 / 8
Adaptive sampling (threshold)
Enabled (0.2)
Caustics (reflective / refractive)
Disabled
Denoising
Enabled
Resolution
512×512
Appendix
Table 4: Fixed Cycles render settings enforced for all renderings. These parameters cannot be overridden by the agent, ensuring deterministic visual output across episodes.
Endpoint
Method
Agent-Visible Behavior
/exec
POST
Executes bpy code on the Blender main thread; returns a JSON with success flag and captured stdout or traceback.
/preview
POST
Renders the current scene from a named camera via Cycles GPU; returns a base64-encoded PNG image.
/scene_info
GET
Exports the full scene descriptor as structured JSON (object names, types, poses, light attributes, shape-key values).
Appendix
Table 5: BlenderHTTP endpoint specifications from the agent’s perspective.
Figure 6: Illustration of behaviors and common failure patterns in MLLMs (GPT-5.2 and Gemini-3.0). Factual errors in the models’ reasoning are highlighted in red.
Figure 7: Full placement-editing trajectories. The examples illustrate how the agent iteratively updates object pose using rendered feedback and executable Blender code.
Figure 8: Full shape-key editing trajectories. The successful case shows progressive identification and adjustment of relevant shape-key parameters, while the comparison case leaves a larger residual.
Figure 9: Full material-editing trajectories. The example illustrates iterative inspection of material parameters and texture-scale adjustment through sandbox tool calls.
Model
Valid rate ↑
Mean visibility ↑
Mean angle ↓
Qwen3-VL-8B Base
13.3
9.0
167.2
Qwen3-VL-8B BlenderFORGE
35.0
31.6
142.8
Qwen3.5-9B Base
10.0
8.0
167.8
Qwen3.5-9B BlenderFORGE
35.0
23.7
143.0
Appendix
Table 10: Held-out camera-reframing results. Valid rate and visibility are percentages; angular error is in degrees. Each row contains 60 evaluations.
Table 11: Qwen3.5-9B per-family specialists (Spec.) and the jointly trained generalist (Gen.) on the BlenderFORGE test split. PL/N-CLIP: ×10−3 ; SR: percent.
Table 12: Thinking comparisons. Placement/Shape [email protected] is in percent; PL is in ×10−3 . “Yes” denotes enabled thinking. The last row is trained; the others are base models.
Group
Pairs
Base wins
SFT+RL wins
Ties
Overall
160
18
108
34
T
80
10
48
22
T+I
80
8
60
12
Placement / T+I
20
0
17
3
Shape-Key / T+I
20
2
16
2
Material / T+I
20
2
15
3
Appendix
Table 13: GPT-5.5 blinded pairwise preferences on Imagine-OOD. The overall row pools both conditions for the same 80 tasks.
Task
Metric
N
ρ
95% CI
p -value
Placement
P-Chamfer
200
−0.504
[−0.627,−0.368]
2.8×10−14
N-CLIP
200
−0.366
[−0.489,−0.237]
1.0×10−7
PL
200
−0.186
[−0.323,−0.049]
0.008
Shape-Key
ShapeKey-L1
200
−0.391
[−0.501,−0.271]
1.0×10−8
N-CLIP
200
−0.605
[−0.696,−0.498]
2.2×10−21
PL
200
−0.477
[−0.587,−0.359]
9.8×10−13
Appendix
Table 14: Spearman correlations between GPT-5.5 ratings and automatic distances. Negative values indicate that higher visual ratings associate with lower distances.
Stage
GPU usage
CPU/RAM
Time
Total compute
Dataset generation
2 RTX 5090 GPUs for scene generation and rendering
16 cores / 256GB
24 hours
48 GPU-hours
SFT
8 RTX 5090 GPUs for unified mixed-task training
64 cores / 1TB
8 hours
64 GPU-hours
AgentLoop-RL
8 RTX 5090 GPUs for training; up to 8 for sandbox rendering
Up to 128 cores / 2TB
10 days
1920 training; at most 1920 sandbox GPU-hours
Evaluation
2 RTX 5090 GPUs per evaluation run
16 cores / 256GB per run
6 hours per run
12 GPU-hours per run
Appendix
Table 15: Resource estimates for the original 8B experiments. Training GPU-hours use allocated cards times elapsed time; sandbox usage is bounded by its maximum allocation. Evaluation is reported per run, not as a total over all models or additional studies.
Asset / software
Use in BlenderFORGE
License / terms treatment
3D-FRONT / 3D-FUTURE
Base scenes and object assets for placement
Used under provider terms; restricted raw assets are not redistributed.
Infinigen
Lighting scenes and material definitions
Code: BSD 3-Clause ; preserve upstream notices and attribution.
Blender / bpy
Scene representation, editing API, rendering, and sandbox execution
Used under Blender GNU GPL terms; integration code retains notices.
Qwen3-14B / Qwen3-VL-8B / Qwen3.5-9B
Shape-key renaming and open-weight agent initialization
Apache 2.0 for these checkpoints; retain license and attribution notices.
GPT-5.2 / GPT-5.5
Teacher trajectories and visual assessment
Provider API terms; no proprietary weights are redistributed.
Gemini
Imagine-OOD goal images and zero-shot baselines
Provider service terms; no proprietary weights are redistributed.
Appendix
Table 16: Primary existing assets, models, and software used by BlenderFORGE. We cite the original sources in the main paper and will include license and terms-of-use notes in the release package.