DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation

Junyi Ouyang; Wenbin Teng; Gonglin Chen; Yajie Zhao; Haiwei Chen

cs.CVMar 25, 2026

DCARL: A Divide-and-Conquer Framework for Autoregressive Long-Trajectory Video Generation

Authors: Junyi Ouyang, Wenbin Teng, Gonglin Chen, Yajie Zhao, Haiwei Chen

Organizations: Institute for Creative Technologies · University of Southern California

Abstract

Long-trajectory video generation is a crucial yet challenging task for world modeling primarily due to the limited scalability of existing video diffusion models (VDMs). Autoregressive models, while offering infinite rollout, suffer from visual drift and poor controllability. To address these issues, we propose DCARL, a novel divide-and-conquer, autoregressive framework that effectively combines the structural stability of the divide-and-conquer scheme with the high-fidelity generation of VDMs. Our approach first employs a dedicated Keyframe Generator trained without temporal compression to establish long-range, globally consistent structural anchors. Subsequently, an Interpolation Generator synthesizes the dense frames in an autoregressive manner with overlapping segments, utilizing the keyframes for global context and a single clean preceding frame for local coherence. Trained on a large-scale internet long trajectory video dataset, our method achieves superior performance in both visual quality (lower FID and FVD) and camera adherence (lower ATE and ARE) compared to state-of-the-art autoregressive and divide-and-conquer baselines, demonstrating stable and high-fidelity generation for long trajectory videos up to 32 seconds in length.

Figures & tables

Figure 1: Given a single Input Frame (left), a camera trajectory and a text description(omitted for brevity), DCARL generates high-fidelity and temporally consistent future scene sequences spanning $32\text{s}$ . The figure showcases synthesis results across three diverse environments: urban (top), snowy suburban (middle), and rural (bottom). The bottom panel shows the corresponding 3D point cloud reconstructions by $\pi_{3}$ [ 24 ] of the generated sequences. The Generated sequences exhibit strong visual realism and long-term temporal coherence that closely match the quality of Reference sequences, highlighting our method’s efficacy for long-trajectory generation.

Figure 2: Our Two-Stage Autoregressive Pipeline for Long-Term Video Synthesis. Our model synthesizes a long video sequence $\mathcal{V}=\{I_{1},\dots,I_{N}\}$ conditioned on multimodal inputs: the initial image $I_{0}$ , camera trajectory $\mathcal{T}$ , and caption $\mathcal{C}$ . The Keyframe Generator $G_{K}$ produces a global Keyframe Set $\mathcal{K}$ , establishing the overall structural evolution of the video. The Interpolation Generator $G_{I}$ generates the final frames in an autoregressive manner.

Figure 3: Qualitative comparison of long-trajectory video generation. We compare our method against several state-of-the-art baselines. Each method is provided with the same initial frame as input. Our approach demonstrates superior long-trajectory consistency and visual fidelity over a 32-second duration.

Method	Overall FID/FVD $\downarrow$	0-8 s	8-16 s	16-24 s	24-32 s	ATE $\downarrow$	ARE $\downarrow$
DiffF	35.0 / 664.1	25.3 / 390.7	39.8 / 684.5	45.2 / 856.8	54.1 / 1067.1	0.469	19.448
SelfF	58.0 / 2113.6	26.2 / 533.5	62.8 / 1911.3	92.8 / 3402.2	119.4 / 4183.8	0.610	14.386
DeepF	42.3 / 1558.5	26.1 / 545.3	49.1 / 1472.3	63.7 / 2342.2	77.2 / 2832.6	0.571	15.144
Vista	66.7 / 1550.0	29.1 / 428.7	66.3 / 1212.6	103.9 / 2452.3	134.1 / 3774.5	0.641	19.332
SEVA	22.2 / 548.0	27.2 / 582.0	30.7 / 643.2	31.8 / 623.8	33.1 / 602.0	0.294	8.527
Ours	19.2 / 203.7	19.6 / 191.4	25.1 / 255.8	27.3 / 276.0	28.6 / 313.8	0.237	7.669

Table 1: Quantitative comparison on 32-second videos of the ODV-YT dataset. Metrics are reported as FID / FVD for visual quality and temporal continuity, and ATE / ARE for global path adherence. Our method maintains superior stability and fidelity over extended durations.

Method	Overall FID/FVD $\downarrow$	0-4 s	4-8 s	8-12 s	12-16 s	ATE $\downarrow$	ARE $\downarrow$
DiffF	37.1 / 566.6	29.3 / 376.3	41.8 / 573.7	51.7 / 738.9	60.9 / 906.2	0.154	13.704
SelfF	53.1 / 711.2	53.1 / 394.8	59.3 / 592.1	69.1 / 989.0	78.7 / 1504.7	0.092	12.464
DeepF	50.3 / 618.4	52.5 / 343.3	59.0 / 577.3	64.7 / 887.2	69.9 / 1174.8	0.104	10.774
Vista	68.7 / 1239.1	27.3 / 445.1	54.2 / 847.0	111.8 / 1860.8	160.5 / 2966.4	0.195	16.607
SEVA	35.9 / 487.9	30.2 / 493.7	41.3 / 529.1	54.7 / 595.2	72.1 / 769.4	0.117	6.289
Ours	19.6 / 225.4	23.3 / 239.9	26.5 / 264.3	31.1 / 323.0	30.8 / 345.2	0.045	5.274

Table 2: Quantitative evaluation of zero-shot generalization on 16-second videos of the nuScenes dataset. Our approach achieves consistent high-fidelity generation and precise camera control in both FVD and ATE.

Figure 4: Qualitative comparison of keyframe model designs. The top row shows the results of the temporal compression baseline, while the bottom row shows our proposed design. Our approach provides significantly clearer visual details and superior structural accuracy compared to the baseline.

Model Design	FID $\downarrow$	ATE $\downarrow$	ARE $\downarrow$
Temporal compression	19.6	0.155	7.276
Ours	16.3	0.100	3.999

Table 3: Ablation study on keyframe model design. We compare our proposed image-level encoding against a temporal compression baseline, which treats keyframes as a video sequence with temporal downsampling. Our design achieves superior performance in both visual quality and trajectory adherence in keyframe generation.

$(\Delta k_{gen},\Delta k_{int})$	(4, 4)	(4, 8)	(4, 16)	(8, 8)	(8, 16)	(16, 16)
FID $\downarrow$	21.4	21.0	20.9	19.3	19.0	18.8
FVD $\downarrow$	201.8	199.9	204.6	196.0	201.5	187.8
ATE $\downarrow$	0.148	0.147	0.131	0.096	0.098	0.104
ARE $\downarrow$	4.98	4.91	5.08	3.96	4.06	4.74

Table 4: Ablation on keyframe intervals for generation (

\Delta k_{gen}

) and interpolation (

\Delta k_{int}

). Keyframes are initially generated at stride

\Delta k_{gen}

and subsequently filtered to

\Delta k_{int}

for dense interpolation.

Figure 5: Ablation on noisy keyframe design. We compare W/o noisy keyframe , Train w/ noisy keyframe , and our full Train/infer w/ noisy keyframe strategy. Applying noise during both stages effectively mitigates memory bias and ensures smoother transitions.

Training Design	FID $\downarrow$	FVD $\downarrow$
W/o noisy keyframe	19.0	246.5
Train w/ noisy keyframe	19.9	240.9
Train/infer w/ noisy keyframe	19.3	201.7

Table 5: Ablation on noisy keyframe design. All methods take the same generated keyframes as input. Applying noisy keyframes during both training and inference achieves the best dynamic video quality (lowest FVD).

Design	MS $\uparrow$	PSNR $\uparrow$	SSIM $\uparrow$
Overlap	0.969	18.15	0.534
+ Sub	0.968	17.95	0.524
+ Sub + Train	0.972	18.75	0.562

Table 6: Ablation on segment continuity design. We compare Overlap (simple overlapping), +Sub (latent substitution), and our full +Sub+Train strategy. Our method effectively bridges segment junctions, achieving the best motion smoothness and structural fidelity.

Figure 6: Ablation on Frame Continuity Design. The designs test continuity at segment boundaries: overlap (simple overlap); + sub (overlap + latent substitution); and + sub + train (our trained continuity mechanism).

Method	Overall FID/FVD $\downarrow$	0-8 s	8-16 s	16-24 s	24-32 s	ATE $\downarrow$	ARE $\downarrow$
w/o Keyframe	25.2 / 376.7	21.1 / 247.1	31.5 / 407.5	38.2 / 546.5	45.2 / 665.3	0.387	12.184
Ours	19.2 / 203.7	19.6 / 191.4	25.1 / 255.8	27.3 / 276.0	28.6 / 313.8	0.237	7.669

Table 7: Ablation study on the effect of keyframes for 32-second videos on the OpenDV-YT dataset. The inclusion of keyframes significantly enhances long-term consistency and trajectory fidelity.

Figure 7: Qualitative ablation of keyframe guidance on 32-second videos. The top row shows the results without keyframe conditioning, and the bottom row shows our full model with keyframe anchors.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Overall	0-8 s	8-16 s	16-24 s	24-32 s	ATE $\downarrow$	ARE $\downarrow$
DiffF	36.5 / 450.1	32.3 / 397.0	45.9 / 576.4	49.8 / 630.1	54.6 / 687.9	0.517	41.467
SelfF	50.4 / 1062.2	46.3 / 571.5	56.1 / 877.7	78.2 / 1569.3	105.2 / 2374.8	0.699	75.676
DeepF	58.0 / 1279.5	37.4 / 479.7	73.0 / 1454.6	83.4 / 1905.0	85.5 / 1988.0	0.664	66.689
SEVA	24.8 / 275.0	35.9 / 290.6	40.8 / 390.0	39.0 / 371.8	38.0 / 397.0	0.670	68.129
Ours	22.8 / 220.1	29.1 / 294.6	33.3 / 344.0	30.1 / 324.1	33.3 / 320.1	0.340	19.646

Appendix

Table A : Comparison of our method with state-of-the-art approaches for 32-second video generation on the DL3DV-10K dataset.

Figure A : Visual comparison of 32-second video generation on the OpenDV-YouTube [ 29 ] dataset.

Figure B : Visual comparison of 16-second video generation on the NuScenes [ 2 ] dataset.

Figure C : Qualitative comparison of 32-second video generation on the DL3DV dataset. Compared to other methods, DCARL preserves better stability and high-frequency details in small-scene environments.

Figure D : Visual analysis of failure cases. Rows 1-2: Challenges in long-distance perception, such as maintaining roundabout structures and respecting pedestrian islands. Row 3: A corner case involving complex structural transitions (exiting an overpass) that leads to unnatural artifacts.

Explore similar work

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation
Sep 29, 2026Byoungwoo Park, Jaemoo Choi, Juho Lee +1Video Diffusion Models Long-Horizon Video Generation
KAIST · Georgia Tech
TRAC: Trajectory-aware Reuse and Adaptive Correction for Efficient Autoregressive Video Generation
Oct 2, 2026Jiaxing Song, Weiqi Yan, You Huang +4Autoregressive Video Generation Inference Acceleration
School of Computer Science and Technology, Hainan University · MAC Lab, Department of Artificial Intelligence, School of Informatics, Xiamen University · Zhejiang Normal University, Jinhua, China
TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment
Jun 11, 2026Yu Meng, Xiangyang Luo, Letian Li +5Long-Horizon Video Generation Diffusion Model Caching
Tsinghua University · D-INFK, ETH Zürich

Sep 29, 2026J/K move · Enter open · S save

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

Byoungwoo Park, Jaemoo Choi, Juho Lee, Yongxin Chen

KAIST · Georgia Tech

Video Diffusion Models Long-Horizon Video Generation Video Generation Autoregressive Video Generation Distribution Matching Distillation Temporal Consistency in Video Generation

Open paper AskPDF arXiv

World models, game simulators, and long-take video creation require coherent scene evolution and sustained dynamics over extended durations. Autoregressive (AR) video diffusion provides a natural framework for long-horizon generation, yet extended rollouts often become near-static or lose visual quality. We hypothesize that these failures reflect the limited guidance provided by short-video supervision on how ongoing scene dynamics develops over longer durations. This motivates us to introduce LongTake, a two-stage training pipeline built around Long-Horizon Teacher Forcing (TF) on curated real long videos. Long-Horizon TF trains the AR model to predict later frames conditioned on long ground-truth video prefixes, extending direct supervision beyond the short training horizon. This supervision is designed to help the model sustain dynamics and preserve visual quality during long-horizon generation. Our central finding is that this training stage strengthens direct initialization for distribution matching distillation (DMD) under student self-rollout, without the intermediate few-step distillation stage used in standard pipelines. Under the same five-second DMD training setup, our initialization yields substantially higher dynamic degree than short horizon TF initialization on 30-second rollouts at comparable aesthetic quality, and surpasses the evaluated baselines in both measures. Hybrid DMD further reuses this teacher to extend supervision to later frames of the self-rollout while retaining bidirectional joint supervision over the initial window. On long-horizon self-rollouts, LongTake lies on the Pareto front of dynamic degree and aesthetic quality, and Hybrid DMD attains the highest dynamic degree among evaluated methods at both 30s and 60s.

Abstract

Explore similar work

LongTake: Learning to Sustain Dynamics in Long-Horizon Video Generation

TRAC: Trajectory-aware Reuse and Adaptive Correction for Efficient Autoregressive Video Generation

TetherCache: Stabilizing Autoregressive Long-Form Video Generation with Gated Recall and Trusted Alignment