CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

Jialei Wang; Zehan Wang; XiaoYang Wu; Ziang Zhang; Siyu Chen; Jiaer Huang; Yujia Zhang; Tianyu Pang

cs.CVOct 12, 2026

CineRecon: Benchmarking Frontier Models for Agentic Cinematic Reconstruction

Authors: Jialei Wang, Zehan Wang, XiaoYang Wu, Ziang Zhang, Siyu Chen, Jiaer Huang, Yujia Zhang, Tianyu Pang,

Organizations: The University of Hong Kong · Zhejiang University · New York University

Abstract

Reconstructing reference videos as editable dynamic 3D scenes is an important path toward controllable video generation for filmmaking: these scenes provide modifiable, renderable visual proxies that enable explicit control over scene layout, subject actions, and camera behavior. To evaluate agents' ability to reconstruct editable dynamic 3D scenes from dynamic reference videos, we introduce CineRecon, a benchmark comprising 50 reference videos and 360 annotated evaluation clips. The reference videos span film, real-world footage, and games. Given a complete video, an agent uses tools to construct, animate, render, and refine its reconstruction. We evaluate this task under three representation settings: Proxy, Simplified, and Detailed. These settings increase geometric and appearance requirements while retaining the same requirements for spatial relations, camera behavior, and subject dynamics. This design tests whether agents can reproduce the events in a video when detailed appearance is unnecessary, and how performance changes as representation requirements increase. For evaluation, we construct checklists that associate observable properties and events with specific entities and times. A vision-language judge compares timestamp-aligned reference and rendered frames across 11 categories, grouped into camera, static, and dynamic consistency dimensions. We evaluate 12 models using their corresponding agent harnesses and find failures across all three dimensions. Subject motions and interactions remain difficult to reproduce even under the Proxy setting.

Figures & tables

Figure 2: Reconstruction performance. Scores for 13 configurations of 12 models across 11 check categories and Overall, averaged equally across Proxy, Simplified, and Detailed. Appearance averages Simplified and Detailed. Overall weights camera, static, and dynamic consistency equally. Low denotes low reasoning effort.

Figure 3: Generation cost and reconstruction performance. Overall score versus mean generation cost per video (USD, logarithmic axis). Scores weight the three consistency dimensions and three representation settings equally. The dashed line connects the observed Pareto frontier. Low denotes low reasoning effort.

Figure 4: Representation settings. Reference frames and Opus 5.5 reconstructions under Proxy, Simplified, and Detailed. Geometric and appearance requirements increase across the three settings, while requirements for spatial relations, camera behavior, motion, interactions, and event timing remain unchanged.

Figure 5: Reference videos and dataset characteristics. (a) Reference sequences illustrating subject actions, motion and interactions, camera motion, zoom and focus changes, and shot transitions. (b) Annotated subjects per video: individually numbered characters, animals, and other main objects, counted once across clips within each video. (c) Reference video duration. (d) Video-level coverage of camera motion, subject interaction, and shot transitions, aggregated from the annotations. Statistics cover all 50 reference videos. The characteristics in (d) may co-occur.

Figure 6: Overall alignment with human preferences. Win rates weight the three settings equally within each dimension, then average camera, static, and dynamic consistency equally. Logo badges identify models. Thin leaders locate overlapping results at their exact coordinates. The dashed line denotes equal win rates.

Figure 7: Spatial, appearance, and motion comparisons. Each case shows the reference, Opus 5.5, and GPT-6 Astra under the indicated setting. Panels (a)–(d) compare corresponding single frames. Panels (e)–(f) show three-frame sequences progressing from left to right at matched times. Gray boxes locate reference details. Green and red local boxes highlight reproduced details and remaining errors, respectively.

Figure 8: Representation effects. Squares show camera and dynamic consistency scores under three settings, with lines connecting each configuration’s scores. Scores average checks within clips, then weight clips equally. Mean weights the 13 configurations equally. Annotations report Proxy minus Detailed, computed from unrounded values.

Figure 9: Action reproduction under Proxy. Six cases compare the reference, Opus 5.5, and GPT-6 Astra, with three time-matched frames progressing from left to right in each row. The examples cover articulated actions, hand–object interaction, steering, and landing. Gray boxes locate reference details. Green and red local boxes highlight reproduced details and remaining errors, respectively.

Figure 10: Iteration and effort. (a) Opus 5.5 scores under three execution configurations, with equal weights across settings and consistency dimensions. (b) Corresponding frames from a Detailed steering sequence, ordered left to right. Gray boxes mark reference details, green boxes reproduced steering, and red boxes the rotating display (No Refine) or absent steering (Low Effort).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Statistic	Value
Reference videos	50
Total reference duration (s)	763.59
Mean reference duration (s)	15.27
Reference duration range (s)	4.90–24.04
Evaluation clips	360
Clips per video, range	3–13

Appendix

Table 1: CineRecon dataset statistics. Videos and evaluation clips are counted once. Check items are counted separately for each applicable representation setting.

Name	First frame	Identification
Person A	0	Long light-colored hair, a beige patterned short-sleeved top, and a long white skirt; seated barefoot on the dirt road, slightly left of center.
Person B	0	Short dark hair, a light-blue short-sleeved collared shirt, brown trousers, and dark-brown shoes; leaning forward and crouching to the image-right of Person A.

Appendix

Table 2: The two numbered subjects in the example clip. First appearances use local frame indices.

Category	Frames	Check item	Rating
Object presence	0	Person A is clearly visible.	$\checkmark$
Object presence	0	Person B is clearly visible.	$\checkmark$
Object appearance	0	Person B has short dark hair and wears a light-blue short-sleeved collared shirt, brown trousers, and dark-brown shoes.	$\checkmark$
Spatial relations	0	Person B is to the image-right of Person A, beside A’s outstretched feet, with knees bent and torso leaning forward; both hands remain above the dirt road.	$\checkmark$
Spatial relations	46	Person B crouches low to the image-right of Person A, beside A’s outstretched feet, with the hand on the image-right supporting against the dirt road.	$\times$
Camera translation	0–46	The camera remains at a fixed position, with no visible horizontal or vertical translation.	$\checkmark$

Appendix

Table 3: Selected Simplified checks and VLM judge ratings for Opus 5.5 on this clip.

\checkmark

: fully satisfied (1);

\times

: unsatisfied (0). None of these eight checks received a partially satisfied (0.5) rating. Statements are translated from the released checklist.

Model	Agent harness	Reasoning
GPT-6 Astra [ 35 ]	Codex CLI 0.153.4	high
GPT-5.6 Sol [ 34 ]	Codex CLI 0.153.4	high
Opus 5.5 [ 5 ]	Claude Code 2.1.280	high/low
Opus 4.8 [ 4 ]	Claude Code 2.1.278	high
GLM-5.3-Flash [ 41 ]	ZCode CLI 0.16.9	high
Qwen3.8-Max [ 3 ]	Qwen Code 0.24.1	xhigh

Appendix

Table 4: Agent harnesses and reasoning settings.

Rating	Score	Criterion
Unsatisfied	0	A required subject is absent, a key fact is missing or reversed, or the main required relation does not hold.
Partially satisfied	0.5	The main fact or salient feature holds, but some required attributes, spatial relations, timing, motion, or contact remain visibly incorrect.
Fully satisfied	1	All observable requirements of the statement hold at the specified frame or interval, allowing the simplifications permitted by the setting.

Appendix

Table 5: Check-item rating rules. Scores reflect agreement with the statement at its specified frame or interval under the assigned representation setting.

Dimension	Proxy	Simplified	Detailed	Total
Camera	161	160	159	480
Static	159	161	160	480
Dynamic	160	159	161	480
Total	480	480	480	1,440

Appendix

Table 6: Human comparison tasks by dimension and representation setting.

Dimension	$N$	Spearman $\rho$	Pearson $r$
Camera	448	0.902	0.959
Static	456	0.965	0.951
Dynamic	456	0.832	0.904
Overall	1,360	0.951	0.975

Appendix

Table 7: Human–automatic alignment. Correlations are computed across all 12 models.

N

counts valid comparison tasks.

Model	Proxy				Simplified				Detailed
	Overall	Cam.	Static	Dyn.	Overall	Cam.	Static	Dyn.	Overall	Cam.	Static	Dyn.
Opus 5.5	76.5	90.5	78.2	60.8	76.9	91.0	79.2	60.4	76.4	91.3	76.6	61.3
GPT-6 Astra	67.6	84.1	74.0	44.6	63.0	79.1	69.3	40.6	60.8	78.5	65.6	38.1
Kimi K3	57.3	74.6	63.2	34.0	51.8	69.7	55.8	30.1	48.2	66.8	49.7	28.0
Opus 5.5 (Low)	49.1	67.9	53.9	25.4	51.4	71.9	55.6	26.7	48.1	67.5	50.3	26.5
DeepSeek-V4.1-Flash	53.9	71.0	63.3	27.6	51.1	68.0	57.3	28.0	43.4	60.6	48.0	21.7

Appendix

Table 8: Scores for 13 model configurations across the three representation settings (0–100). Dimension scores weight clips equally. The overall score averages the three dimensions. Configurations are ordered by their mean overall score across settings. The mean row weights configurations equally. Low denotes low reasoning effort.

Contrast	Proxy	Simplified	Detailed
Motion $-$ action	22.2	18.3	15.7
Target $-$ rotation	35.0	33.1	32.1
Presence $-$ relation	28.3	23.4	22.4
Presence $-$ appearance	—	21.3	30.5
Static $-$ dynamic	28.6	25.7	21.2

Appendix

Table 9: Differences between category and dimension means, averaged equally across 13 configurations (points). Appearance checks are not applicable to Proxy.

3D Reconstruction Benchmarks

See all 15 papers

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender
Sep 14, 2026Yolo Y. Tang, Daiki Shimada, Jiayue Meng +14AI Agent Benchmarks Temporal Video Understanding
University of Rochester · Sony Group Corporation · Carnegie Mellon University +1
4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes
Oct 2, 2026Ruihong Shen, Žiga Kovačič, Peter Kulits +6Dynamic Scene Reconstruction AI Agent Benchmarks
Johns Hopkins University · Stanford University · Max Planck Institute for Intelligent Systems +1
RenderBench: Benchmarking Render-to-Real Video Transfer with Reconstructed Digital Twins
Oct 6, 2026Dicong Qiu, Zhiyuan Xu, Yaosheng Liu +2Sim-to-Real Transfer Digital Twins
The Hong Kong University of Science and Technology (Guangzhou) · Southeast University · Independent Researcher

Sep 14, 2026J/K move · Enter open · S save

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

Yolo Y. Tang, Daiki Shimada, Jiayue Meng, Jing Bi+13

University of Rochester · Sony Group Corporation · Carnegie Mellon University+1

AI Agent Benchmarks Temporal Video Understanding

Open paper AskPDF arXiv

Multimodal agents can create complex videos in software such as Blender by writing code instead of using diffusion models. Yet video understanding benchmarks still evaluate models mainly through question answering. If an agent truly understands a video, it can reconstruct it programmatically. We introduce BVB, Blender-VideoBench, a benchmark that tests this ability by asking agents to reconstruct real-world videos as animated Blender scenes. To ensure fair comparison, each agent programs the reconstruction through a lightweight harness, Mini-BVB, in an identical sandbox under a shared cost limit. The benchmark renders each reconstruction from its animated camera and evaluates it on two axes: (1) Dual VQA measures how many spatiotemporal facts the reconstruction preserves. (2) Latent Similarity measures how closely the reconstruction matches the source video perceptually. Our overall score, a square-root mean, favors balanced performance. We evaluate 51 configurations from 10 model families and analyze semantic retention, perceptual similarity, reasoning effort, and cost. The best model reaches 88.6 Latent Similarity but retains only 53.7% of the spatiotemporal facts from the source video. Additional reasoning improves perceptual similarity but does not close this gap. In a blind study with 15 raters and five configurations, Latent Similarity correlates strongly with human preference. These results show that programmatic reconstruction is a viable test of agentic video understanding, and that semantic retention remains the main challenge.

Abstract

3D Reconstruction Benchmarks

Explore similar work

BVB: Benchmarking Agentic Video Understanding via Programmatic Reconstruction in Blender

4DCodeBench: Benchmarking Agents on Inverse Graphics of Dynamic Scenes

RenderBench: Benchmarking Render-to-Real Video Transfer with Reconstructed Digital Twins