cs.CVDate pending

Streaming4D: Accelerate 4D World Models via Block-wise Video Generation and Incremental Reconstruction

Authors: Xiaoyan LiuJiaxin LiuKangrui LiSifan Zhou

Organizations: 1The Chinese University of Hong Kong · 2The Hong Kong Polytechnic University · 3The University of New South Wales

Abstract

Current 4D generation paradigms are often bottlenecked by a sequential decoupling design: video is generated first, followed by 3D reconstruction, leading to high interaction latency. This limits applications in interactive real-time scenarios. To this end, we propose \textbf{Streaming4D}, a tightly coupled synchronous pipeline that integrates block-wise autoregressive video generation with incremental 3D reconstruction. Unlike traditional frame-by-frame emission and delayed geometry recovery, Streaming4D generates temporal video blocks and immediately triggers reconstruction for each completed block, enabling parallel execution between synthesis and geometric updates. This approach allows the world representation to evolve online with the video stream, reducing feedback latency while preserving geometric fidelity. We instantiate \textbf{Streaming4D} using a Self-Forcing-style autoregressive generator and an incremental reconstruction backend. Experiments show consistent runtime improvements across resolutions on a single RTX 4090 (1.24×\times speedup), while maintaining high-quality 4D geometry and multi-view consistency.

Explore similar work

CardsList
  1. Beyond Pixels: From Video Priors to 4D Worlds

    Aug 11, 2026Zihao Liu, Xiaolong Shen, Zhenglin Zhou +24D GenerationVideo Latents