cs.AINov 7, 2026

BlenderFORGE: Framework for Optimizing Reactive 3D-Graphics Editing Ability of MLLMs

Authors: Zilin Guo, Tianrui Zhang, Yi Rong, Yichen Liu, Yuxin Guo, Jingcheng Ni, Lewei Lu, Zehuan Wu, 

Organizations: The Hong Kong University of Science and Technology · SenseTime Research

Abstract

Multimodal Large Language Models (MLLMs) have shown promise as interactive agents, yet precise, programmatic 3D graphics editing remains difficult to train because existing systems largely rely on closed-source model APIs and lack deterministic, state-verifiable execution environments. We present BlenderFORGE, a trainable data-environment-optimization framework for open-weight MLLM agents that edit explicit Blender scenes through Blender Python (bpy) code generation. BlenderFORGE targets four fundamental state-verifiable editing task families: object placement, shape-key editing, lighting adjustment, and material modification. It integrates three components: (1) a scalable perturbation-based data generation pipeline built from curated assets and large-scale 3D scene repositories; (2) an isolated Blender Sandbox for deterministic code execution, structured state export, and closed-loop reward computation; and (3) a two-stage training pipeline that uses offline teacher trajectories for cold-start Supervised Fine-Tuning (SFT) and sandbox-computed rewards for multi-turn tool-agentic Reinforcement Learning (AgentLoop-RL). Experiments across the four task families show that the resulting open-weight Qwen3-VL-8B and Qwen3.5-9B BlenderFORGE agents substantially improve over their base models and achieve competitive performance with several zero-shot proprietary MLLM baselines on selected metrics. Additional cross-benchmark evaluations and camera-reframing experiments provide evidence of transfer beyond the original training setting.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. ViSculpt: Visual-Centric Agentic Geometry Editing

    Aug 25, 2026Bo Pang, Jiaqi Pan, Xiaocheng Zhang +33D Shape EditingGUI Agents

  2. 3DCodeEdit: Benchmarking Instruction-Guided 3D Program Editing

    Oct 28, 2026Binjie Wang, Xinran Zhao, Shoubin Yu +83D Asset Editing3D Scene Editing

  3. BlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World References

    Oct 14, 2026Xinyu Fang, Yuchi Wang, Weihao Yu +123D Scene Generation3D Reconstruction Benchmarks