While frontier coding agents can generate high-quality 3D scenes and articulated assets, they struggle to create coherent long-horizon animations. When equipped with general-purpose harnesses, they frequently produce physically implausible outputs, generating objects that interpenetrate, lack physical support, or move unnaturally. Such bare agents struggle to repair these errors because they default to scripting the full animation in one pass, making it difficult to verify their work and disentangle complex spatiotemporal constraints. Drawing on the practice of human animators, we introduce DyCraft, a system for animated 3D scene generation that breaks animation into steps that coding agents can more reliably solve. DyCraft first expands the user's prompt into a detailed plan and builds a static scene. It then creates a 3D storyboard of key animation poses, where each frame is a static 3D scene that the agent can build and debug in isolation. Storyboarding decomposes the problem into pairs of verified start and end states, allowing the agent to author local motion between them with a collection of motion authoring tools. We evaluate DyCraft on 24 long-horizon, multi-shot animation tasks covering humanoid robots, human and animal characters, and physical mechanisms. Compared with a native GPT-6 agent in Codex and prior agentic baselines, DyCraft substantially reduces physical violations and generates higher-quality animated 3D scenes that are more often preferred by human raters.
Figures & tables
Figure 1: DyCraft is an end-to-end agentic system for generating long-horizon animated 3D scenes from text prompts. Our system allows GPT-6-Astra coding agents to write minute-long 3D animations in Blender, depicting humanoid robots, human characters, and physical mechanisms. 3D moments from the authored films are shown above. Project page: https://dy-craft.github.io
Figure 2: Failure modes of bare agents. The top row shows GPT-6-Astra with Codex producing interpenetration, missing contact, grasp failure, motion discontinuity, unusable layout, and faked dynamics. The bottom row shows corresponding results produced by DyCraft with GPT-6-Astra.
Figure 3: Overview of DyCraft on an example. A humanoid robot carries a wide panel through a narrow doorway. Following pose-to-pose animation, DyCraft breaks authoring into steps that the agent can check individually. The sparse user prompt (1) is first expanded into detailed scene and motion briefs (2), which the agent uses to build the scene in CAD (3). The agent then plans key poses and object states as a 3D storyboard (4). After inspecting and verifying these states, the agent authors motion connecting them with domain-specific tools (5). Verifiers check every frame (6), catching errors in the storyboard states and motions: here, the arm passes through the torso (orange). The agent repeatedly revises the scene and motion before rendering the final film (7).
Figure 4: Scene authoring with CAD tools. Scene components authored by DyCraft (top) and by a bare agent (bottom) for a marble run (both GPT-6-Astra). CAD operations allow DyCraft to construct precise solid geometry, whereas the bare agent approximates the same components with primitives and thickened surfaces that intersect or fail to connect.
Figure 5: Motion authoring tools. Motion authored by DyCraft (top) and by a bare agent (bottom) for the same prompts. Using domain-specific techniques such as physical simulation and inverse kinematics, DyCraft produces motion that stays physically plausible, avoids interpenetration, and responds to dynamics. The bare agent’s hand-scripted motion often violates these constraints. Orange marks bare-agent defects and blue arrows indicate time.
Figure 6: Verifiers. Each column shows a state that fails a verifier (top). After agent repair, the verifier passes (bottom). Verifiers for interpenetration, support, and joint limits catch errors that are easily missed by visual inspection, such as characters passing through furniture, objects floating above their supports, and joints that violate anatomical limits or rotate about a wrong axis.
Figure 7: Qualitative comparison with VIGA and a bare agent. Please view our video results on the supplemental webpage. Each column corresponds to an event described in the prompt and shows the frame at which it occurs in each method’s animation; events that a method does not produce are marked as not performed. Compared with DyCraft , the bare agent produces simpler scenes with physically implausible motion and VIGA fails to generate several key events. Translucent overlays show earlier poses, trails indicate fast motion, and the main actor is highlighted in blue.
Table 8
Figure 8: Ablation examples on barista making coffee. For each variant, we show defects over DyCraft on the same action.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Ours vs. VIGA (%)
Ours vs. Bare Agent (%)
Prompt
4D
Physical
Motion
4D
Physical
Motion
A humanoid cooks scrambled eggs, fridge to plate
91
82
91
78
89
78
Flat-pack office chair assembly
100
100
100
91
100
100
Loading a pottery kiln
92
91
91
80
80
60
Mopping a room toward the exit
91
91
91
100
91
82
Carrying a wide panel through a doorway
90
90
90
82
73
64
Appendix
Table 3: Detailed human preference results. Percentage of binary-choice judgments favoring DyCraft over each baseline for each prompt in DyBench . The three aspects are overall 4D realism, physical plausibility, and motion quality. We recommend viewing these results alongside Fig. B.1 and the videos.
Figure S2: Overview of DyBench. The 24 DyBench scenes generated by DyCraft, arranged on a 5×5 board. Colored curves show the trajectories of key moving objects in each scene.
Frontier coding agents can now write and execute code that authors 3D environments, but whether they reliably understand 3D structure and precisely control scene state remains unclear. The generated 3D scene is a persistent, executable artifact: a convincing render can hide incorrect spatial relations, intersecting objects, or unintended modifications. We introduce Code4Scene, a benchmark of 170 Unreal Engine cases built from human-assembled scenes that evaluates coding agents on two complementary settings under a shared execution interface. Construction tests scene-level spatial reasoning from open-ended language specifications, where many realizations are valid; editing tests precise control of scene state, where the agent must recover the target scene from reference images while preserving everything else. Rather than scoring code or rendered views, Code4Scene evaluates the generated engine-native scene for task fulfillment, artifact integrity, and static physical validity, with edits additionally compared against withheld ground truth. Across 14 coding-agent systems on the 83-case public set, construction and editing performance are strongly correlated but not interchangeable (Spearman ρ=0.67): Claude Fable 5.1 leads construction, Gemini 3.8 Flash leads editing. Spatial Composition is the weakest construction category for every agent, while editing remains imprecise: the best Repair F1 is only 0.562, and 34.2% of edits that fully recover the target still introduce unintended changes elsewhere in the scene. These results expose a gap between plausible 3D generation and reliable spatial reasoning and state control. The project webpage is at https://simworld.org/code4scene.
Xiaokang Ye, Siddhant Hitesh Mantri, Zimeng Chen +6
Controllable real-world video generation models can condition on a rendered graybox proxy video, which fixes layout, motion and camera. Coding agents can build such videos by reconstructing a 4D scene in Blender, yet no benchmark checks both these videos and the long, multi-shot scenes behind them. We introduce VibePrevisBench, a benchmark for 4D scene reconstruction: an agent turns a brief into an executable 4D scene and renders its proxy video. Its 242 tasks span four input kinds, including scripts, photos, continuations, and reference videos. Each delivery is scored along two evaluation dimensions. Rule-based Scene Check re-runs the code and compares the exported 4D scene with hidden ground truth facts in nine dimensions. VLM-based Screen Check asks a VLM judge whether the facts that pass can be recognized in the proxy video. We analyze the evaluation results and trajectories of 11 frontier coding agents and identify three key observations. First, multimodal understanding separates the agents, especially in understanding space and camera motion. Second, all agents seem to already master basic camera work and staging, and fail at challenging requests. Third, agents inspect their own renders but fail to correct their errors. Agents thus need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief.
Coding agents combine visual understanding with program execution, offering a new route to automated 3D creation. By translating images and instructions into editable programs, they bring reconstruction, generation, and editing into a shared workflow. Yet existing benchmarks fragment evaluation across task scopes and protocols, leaving their capabilities across construction levels unclear. We propose Frontier3DBench, a unified benchmark of 145 tasks across 9 categories spanning object, scene, and interactive open-world creation, with joint evaluation of appearance, structure, and behavior. Experiments with 11 model-harness configurations reveal strengths that vary by level: GPT-6.1-sol leads the object and scene levels and achieves the highest overall score of 66.02/100, whereas GPT-6-astra leads interactive world construction. However, all configurations score lower on reference reconstruction than on text-driven generation in object and scene tasks. Producing plausible content thus remains easier than reproducing specified geometry, layouts, and visual details. Moreover, generated worlds average 83.08/100 on local mechanisms but only 20.82 on motion clearance and 30.26 on stateful workflows: individual mechanisms often function, whereas complete activities that require coordinated actions and persistent state remain difficult. These results identify reference fidelity and coordinated activity completion as central bottlenecks for coding agents in 3D creation.