Controllable real-world video generation models can condition on a rendered graybox proxy video, which fixes layout, motion and camera. Coding agents can build such videos by reconstructing a 4D scene in Blender, yet no benchmark checks both these videos and the long, multi-shot scenes behind them. We introduce VibePrevisBench, a benchmark for 4D scene reconstruction: an agent turns a brief into an executable 4D scene and renders its proxy video. Its 242 tasks span four input kinds, including scripts, photos, continuations, and reference videos. Each delivery is scored along two evaluation dimensions. Rule-based Scene Check re-runs the code and compares the exported 4D scene with hidden ground truth facts in nine dimensions. VLM-based Screen Check asks a VLM judge whether the facts that pass can be recognized in the proxy video. We analyze the evaluation results and trajectories of 11 frontier coding agents and identify three key observations. First, multimodal understanding separates the agents, especially in understanding space and camera motion. Second, all agents seem to already master basic camera work and staging, and fail at challenging requests. Third, agents inspect their own renders but fail to correct their errors. Agents thus need better visual understanding of space and camera, explicit checks for challenging requests, and item-by-item verification of renders against the brief.
Figures & tables
Figure 2: Two complementary evaluation dimensions. Each panel compares a reference video with strong and weak agents’ proxy videos and the corresponding real-world videos generated by Seedance 2.5. (a) Rule-based Scene Check evaluates the 4D scene, and detects the weak agent’s high camera and missed turns. (b) VLM-based Screen Check assesses visibility and recognizability: the weak agent’s car passes the in-frame and turning checks but is too distant to recognize clearly. The real-world videos inherit their proxy videos’ errors. In addition, we note that the generated real-world videos keep the errors of their proxy videos, demonstrating the importance of proxy video generation quality.
Input
Staging
Filming
Evaluation
Benchmark
Text
Image
Video
#Tasks
Duration (s)
People
Interaction
Timed actions
Camera
Multi-shot
4D scene
vs. GT
On screen
Checks/task
Real-world video generation
VBench-2.0 [ 70 ]
✓
✗
✗
–
–
✓
∘
✗
∘
∘
✗
✗
✓
–
WorldScore [ 12 ]
✓
✓
✗
3k
–
✗
✗
✗
✓
✗
✗
∘
✓
–
FilmBench [ 56 ]
✓
✓
✗
1.2k
–
✓
∘
✗
∘
✓
✗
✗
✓
–
PROWBench [ 28 ]
✓
∘
✓
170
5
✓
✓
✓
✓
✗
✗
∘
✓
–
Table 1: Comparison with related benchmarks. Staging : the scene has people, interaction between people or with objects, and actions at given times. Filming : camera placement or motion is specified, and several shots are joined by cuts. Evaluation : the model builds a 4D scene that changes over time; it is checked item by item vs. GT ; the frames are scored on screen ; checks/task counts scored items per task. Duration : target length in seconds, the median for ours. ✓ yes, ∘ partial, ✗ no, – not reported. BVB reuses the videos and questions of VSI-Bench. Only VibePrevisBench is ✓ in every column.
Figure 3: Overview of VibePrevisBench. The 242 tasks span four input kinds (inner ring) and seven scene domains (outer ring), with spans proportional to task counts. Each card is a real task with its input and a frame of GPT-6 Astra’s proxy video.
Figure 4: One task through the evaluation pipeline (video task M50_049, solved by GPT-6 Astra). The agent writes Blender Python in a sandbox and inspects its own renders. We re-run the delivered program from scratch. The replay exports the 4D scene S , the tagged boxes, cameras and cuts of every frame, and renders the proxy video V from it. Rule-based Scene Check compares S with the hidden GT facts, two of 805 checks shown; VLM-based Screen Check asks the VLM judge about V , only for items the rules passed, two verdicts shown.
Figure 5: How VibePrevisBench is built. (1) Scan datasets and self-collected videos are curated with expert models and a VLM into seven scene domains and 33 places. (2) Scripts are written spec-first and must survive two blind readers; humans select and trim clips and screen photo sets. (3) Hidden GT facts come from the scene spec, 4D reconstruction or scanned 3D boxes, and humans review the resulting checks in nine dimensions.
Figure 6: Task statistics , stacked by input kind; dashed lines mark medians. (a) Brief length in words; script lengths cluster near 600, 800 and 1,200 words. (b) Output length; photo tasks show each photo for 12 frames, then sweep the place for 8 s. (c) Number and share of tasks with each staging demand, split by input kind. (d) Rule checks per task, log scale; one fact can expand into several checks, such as pairwise event orders.
Table 2: Main results on 242 tasks. Rule-based Scene Check and VLM-based Screen Check per input kind and over all tasks ( All ); VLM-based Screen Check uses the Gemini 4 judge, which rates only recognition on photos; Overall is their mean. Failed runs score 0. Per column, the best score is in bold and the second best is underlined . The reasoning effort of each agent is given in gray.
Figure 7: Cost vs. score. Mean API cost per task at list prices (log scale) against the overall score; the staircase is the cost–score frontier.
Figure 8: Where agents improve, and where they do not. (a) Rule-based Scene Check gain of the second model over the first in each pair, per input kind, paired over tasks; whiskers show 95% bootstrap intervals. The top two pairs are successive generations of one family, while the bottom pair is a step up in tier within one generation at about five times the cost. (b) Picture-structure score per agent and input kind, which matches each shot’s depth, normals and segmentation to the reference frames; filled dots mark the four agents with the highest overall score. (c) Error rates of the best and the weakest agent, the other nine in gray; ρ is the Spearman correlation with the overall score, and Sec. C.2 defines each rate.
Figure 9: How agents look, judge and code. (a) Score relative to the other agents’ task mean, by third of an agent’s runs ranked by images viewed. (b) Error rate in runs where the agent says its render matches or looks right, minus runs where it does not, with agent and task held fixed; whiskers show 95% bootstrap intervals and rate is the mean error rate. (c) Share of runs calling Action.fcurves , removed in Blender 5. Rows sorted by overall score.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 10: Hidden ground truth facts for a video task. The eight pipeline steps on one clip: shots, registration and per-shot observation, masks, depth and cameras, bodies, lifting to 3D, review and events, and the final checklist of facts with their tolerance windows.
Figure 11: The four kinds of input. One task of each kind. Top: what the agent receives (a script excerpt, a photo, an opening frame, a reference video); photo, continuation and video tasks also come with a text brief. Middle: some of the hidden GT facts for that task, each tagged with its rubric dimension. Bottom: a frame of the proxy video delivered by GPT-6 Astra, with its Rule-based Scene Check score. Timed facts carry tolerance windows, and facts the ground truth cannot verify never count.
Agent level
Run level
Pair
Runs
Spearman ρ
Pearson r
Spearman ρ
Gemini 4
GPT-6.1 Sol
2,628
0.96
0.98
0.76
Gemini 4
Qwen3.8 Omni Flash
2,648
0.94
0.97
0.73
GPT-6.1 Sol
Qwen3.8 Omni Flash
2,628
0.96
0.99
0.79
Human
Gemini 4
242
0.92
0.94
–
Human
GPT-6.1 Sol
242
0.92
0.91
–
Appendix
Table 4: Agreement between judges and with humans. Agent level : correlation of the 11 agents’ mean VLM-based Screen Check scores; run level : Spearman ρ over single runs. Failed runs and the runs GPT-6.1 Sol declined are excluded. Human rows correlate each agent’s win rate in the human study with its mean score from that judge. Mean : each judge’s mean score over all 2,662 runs, failed runs counting 0.
Change
Tasks
n
Δ Scene
Δ Screen
Δ Overall
∣Δ∣
Cost
Rerun
Script
8
− 2.0 ± 5
+ 2.5 ± 14
+ 0.2 ± 7
5.5
0.57 → 0.70
Oracle caption †
Video
5
+ 14.0 ± 8
+ 17.2 ± 13
+ 15.6 ± 6
15.6
1.94 → 0.47
No images
Script
8
− 0.1 ± 4
− 1.6 ± 16
− 0.8 ± 6
5.5
0.57 → 0.59
Appendix
Table 5: Ablations on DeepSeek V4.1 Flash. Each row reruns the agent with one change and reports the paired per-task difference to the main run (mean ± s.d.). Rerun : the same tasks run again unchanged. Oracle caption : video tasks with the video replaced by a text description. No images : every image returned to the agent is replaced by a text placeholder. ∣Δ∣ : mean absolute change of the overall score per task. Cost: agent pertask,reference\rightarrow$ ablation. † The caption is generated from the hidden GT facts (events, cuts, camera), so this row bounds what a perfect perception step would add.
Figure 12: Capability profiles of all 11 agents. Rule-based Scene Check score per dimension (0–100, failed runs count 0) on (a) script tasks and (b) media tasks. Exact values per agent are in Fig. 13 .
Figure 13: Rule-based Scene Check score per dimension (failed runs count 0; column means in brackets; best per column in bold). (a) Script tasks. (b) Media tasks (photos, continuations, videos). Rows sorted by Rule-based Scene Check score over all tasks.
Figure 14: Shared failure modes. (a) Error rates on a challenging request (red) and a common one (green), averaged over the 11 agents; whiskers show the best and worst agent. Camera angle and camera move : shots whose angle or move type differs from the reference. Shot count : video and continuation runs with more or fewer shots than the reference, an upper bound since the reference misses some cuts. Set dressing : objects missing from the scene. Path length : median ratio of agent path to GT path, pooled over all input kinds. (b) Shots that the rules mark wrong for all 11 agents: the reference frame and two agents at the same instant; tags describe what each one shows.
Step
Script (94)
Photo (39)
Cont. (23)
Video (86)
All (242)
GPT-5.6 Sol → GPT-6.1 Sol
+ 3.6 [2.2, 4.9]
+ 11.1 [8.8, 13.5]
+ 5.1 [2.6, 7.6]
+ 6.9 [5.6, 8.2]
+ 6.1 [5.2, 7.0]
Gemini 3.8 Flash → Gemini 4 Argon
+ 3.4 [0.4, 5.8]
+ 14.1 [11.4, 17.1]
+ 7.1 [4.1, 10.5]
+ 9.7 [8.3, 11.0]
+ 7.7 [6.3, 9.0]
GPT-6.1 Sol → GPT-6 Astra
+ 1.2 [ − 0.2, 2.7]
+ 0.5 [ − 1.1, 2.1]
+ 1.7 [ − 0.2, 3.5]
+ 0.9 [ − 0.3, 2.0]
+ 1.0 [0.3, 1.8]
Appendix
Table 6: Within-family Rule-based Scene Check gains per input kind, paired over tasks, with 95% bootstrap intervals. Task counts are given in brackets.
Figure 15: What makes a reference video hard. The 86 video tasks fall into three bins along five properties of the reference clip, task counts under each bin. Lines give the mean Rule-based Scene Check score of the top four agents by overall score and of the other seven; ρ is the Spearman correlation between the property and the task’s mean score over all 11 agents. Failed runs count 0.
Figure 16: Event completion. Each hidden GT fact is triangulated by both scorers: the rules pass it and the VLM confirms it on screen, confirms it partly, or rejects it; or the rules already fail it. Facts without a usable VLM verdict are excluded. (a) Events per agent: confirmed events fall from 54% (GPT-6 Astra) to 18% (Muse Spark 1.3), and weaker agents more often stage an event the rules accept but the screen does not show. (b) Pooled over agents, entities almost never fail, while events do.
Agent
Homes
Offices
Streets
Shops
Industry
Sports
Nature
All
(34)
(27)
(42)
(19)
(31)
(46)
(43)
(242)
GPT-6 Astra
78.3
78.9
74.4 ▼
80.7 ▲
77.0
79.2
77.3
77.7
GPT-6.1 Sol
78.3 ▲
76.5
72.4 ▼
77.3
76.7
76.5
76.0
76.1
Gemini 4 Argon
75.4
74.9
73.6
76.4
75.1
75.3
75.1
75.0
Opus 5.5
76.6 ▲
75.9
70.3 ▼
74.4
75.2
74.4
73.4
74.1
GPT-5.6 Sol
70.4
70.5
66.9 ▼
72.8 ▲
70.9
70.5
71.1
70.2
Appendix
Table 7: Overall score by scene domain. Mean of (Scene+Screen)/2 over the tasks of each of the seven domains of Fig. 3 , task counts in parentheses. Per column, the best score is in bold . ▲ and ▼ mark domains where an agent scores at least 2 points above or below its own All .
Agent
V1 Recognition
V2 Action
V3 Visibility
V4 Story
Screen
(242)
(202)
(203)
(202)
(242)
GPT-6 Astra
91.7
85.6
88.7
68.6
85.4
GPT-6.1 Sol
91.1
82.3
86.3
64.4
83.1
Gemini 4 Argon
87.9
81.2
83.7
66.8
81.7
Opus 5.5
88.8
76.7
82.9
63.1
79.8
GPT-5.6 Sol
89.9
69.7
80.3
58.1
77.5
Appendix
Table 8: VLM-based Screen Check per dimension (Gemini 4 judge). Each dimension is the mean over the tasks that exercise it, task counts in parentheses; failed runs count 0. Screen is the score of Tab. 2 . Per column, the best score is in bold . Figure 13 gives the Rule-based Scene Check dimensions.
Figure 17: Case gallery: homes, offices and schools, shops and hotels, sports and arts. Each domain shows the task input, a reference-video frame, the first of five photos or the opening frame of a continuation, then the frame at the same moment from each agent’s proxy video, agents in the order of Tab. 2 . Each frame carries the agent’s overall score on that task, the best per task in bold. Each task has the highest Kendall τ between its agent ranking and the overall ranking among the media tasks of its domain on which all 11 runs succeed, ties broken by the larger score spread. Frame times are reference keyframes with people or action in view.
Figure 18: Case gallery: industry and transit, streets and cities, nature and water. Layout and task selection as in Fig. 17 .
Agent
Turns
Tools
Min.
In (M)
Out (k)
Cost ($)
Failed
GPT-6 Astra
71
70
93
11.2
149
32.59
0
GPT-6.1 Sol
62
60
105
10.7
195
6.71
0
Gemini 4 Argon
49
85
73
5.8
181
4.45
2
Opus 5.5
47
61
54
6.7
137
7.82
0
GPT-5.6 Sol
82
81
39
6.8
54
6.19
0
DeepSeek V4.1 Flash
131
141
91
27.9
261
1.89
0
Appendix
Table 9: Process and cost per task on all 242 tasks. Means of turns, tool calls, wall-clock minutes, input tokens (millions, cache hits included) and output tokens (thousands, reasoning included); cost in US dollars at the official list prices of late October 2026, computed per request from the API relay logs. Failed : runs without a valid scene after re-execution, counted as in Tab. 2 . Agents in the order of Tab. 2 .
Figure 19: Effort vs. score. Mean turns and output tokens per task against the Rule-based Scene Check score (all tasks), one marker per agent (stars: best, median and weakest); Spearman ρ over the 11 agents.
We introduce 4DCodeBench, a benchmark for 4D inverse graphics through code generation, in which agents reconstruct dynamic scenes from video as executable graphics programs. To accomplish this, agents must translate visual observations into compact representations of scene structure and dynamics, by implementing abstractions such as physical simulations to reproduce complex behavior. To evaluate this capability, we curate a set of real-world videos and construct synthetic scenes spanning diverse physical phenomena, including deformation, fluid flow, and fracture. We perform extensive benchmarking of frontier models, finding that strong static reconstruction capabilities do not yet translate into reliable reconstruction of complex dynamics. 4DCodeBench provides a testbed for tracking progress toward agents that can interpret the dynamics of the world through code. Our benchmark is available at https://github.com/4DCodeBench/4DCodeBench
Ruihong Shen, Žiga Kovačič, Peter Kulits +6
Johns Hopkins University · Stanford University · Max Planck Institute for Intelligent Systems +1
Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes provide modifiable, renderable visual proxies that enable explicit control over scene layout, subject actions, and camera behavior. To evaluate agents' ability to reconstruct editable dynamic 3D scenes from dynamic reference videos, we introduce CineRecon, a benchmark comprising 50 reference videos and 360 annotated evaluation clips. The reference videos span film, real-world footage, and games. Given a complete video, an agent uses tools to construct, animate, render, and refine its reconstruction. We evaluate this task under three representation settings: Proxy, Simplified, and Detailed. These settings increase geometric and appearance requirements while retaining the same requirements for spatial relations, camera behavior, and subject dynamics. This design tests whether agents can reproduce the events in a video when detailed appearance is unnecessary, and how performance changes as representation requirements increase. For evaluation, we construct checklists that associate observable properties and events with specific entities and times. A vision-language judge compares timestamp-aligned reference and rendered frames across 11 categories, grouped into camera, static, and dynamic consistency dimensions. We evaluate 12 models using their corresponding agent harnesses and find failures across all three dimensions. Subject motions and interactions remain difficult to reproduce even under the Proxy setting.
Jialei Wang, Zehan Wang, XiaoYang Wu +6
The University of Hong Kong · Zhejiang University · New York University
Diffusion video models produce striking clips, but they cannot reliably render an exact formula, a correct chart, a rule-consistent game or narration synchronized to on-screen events. A different route is emerging: multimodal agents that write code to render every frame and synthesize every sound. We introduce VibeVideoBench, a benchmark for evaluating agents on code-driven video generation. Its 239 self-contained tasks in English and Chinese span 12 categories, from data stories and music-driven pieces to classroom lessons and paper explainers; each asks an agent, running in its native harness inside a sandbox with no image or video generation model, to deliver a finished MP4 with sound within one hour. Evaluation is hierarchical and evidence-based: program checks of the delivery spec, about 13k task-specific rubric checks traced to the requirements they enforce, and five quality dimensions anchored per task, scored by three VLM judge panels. Across nine frontier agent systems and 2,151 scored rollouts, GPT-6 Astra and Claude Opus 5.5 rank first and second with both the judges and human annotators, and the strongest system satisfies every core requirement on 76.3% of tasks. Delivery is a second axis: counting missing videos as zero changes the ranking, and three quarters of the lost videos are final renders that started too late, ran too slowly or never started, a failure of how agents plan their hour. Agents build their own rendering pipelines and inspect their own frames, yet how much they look does not predict quality. The judges' ranking of the systems closely matches blind human preferences.