Position: Next-Generation Game Engines Should Be Built on Interactive Generative Video
Organizations: HKU · HKUST · Kuaishou Technology
Abstract
Modern game development faces significant challenges in creativity and cost due to predetermined content in traditional game engines. Recent breakthroughs in video generation models, capable of synthesizing realistic and interactive virtual environments, present an opportunity to revolutionize game creation. This position paper argues that next-generation game engines should be built on Interactive Generative Video (IGV). We term such engines Generative Game Engines (GGE), with IGV as the perceptual and world-generation core under designer control. We argue why IGV is uniquely suited for this role, identify what is missing, and examine progress across six functional modules to show how each supports our position, culminating in a five-level maturity roadmap (L0-L4). Drawing on discussions with researchers, game developers, and community participants, we present nine alternative views and ethical considerations to stimulate debate.
Figures & tables
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
| Level | Name | Technical Features | Application Examples | Category |
| L0 | No AI-Assisted Assets Generation | Manual creation and integration of all game assets and logic. | Super Mario : fixed levels; Tetris : fixed rules. | Traditional Manual Game Development |
| L1 | AI-Assisted Assets Generation | AI-assisted creation and integration of game assets and logic. | Cyberpunk 2077 : AI-generated assets; AI Dungeon : real-time NPC dialogues. | |
| L2 | Physics-Compliant Interactive World Generation | Real-time physics-compliant video generation with user interactions , supported by the Dynamics module. | E.g., Player sets fire to wooden bridges, AI dynamically renders blazing spans and rerouted enemy paths | Next-Gen AI-Driven Generative Game Engine |
| L3 | Causal-Reasoning World Simulation | World simulation with causal reasoning across time based on L2, incorporating the Intelligence module. | E.g., Killing a faction leader in Act 1 triggers city-wide riots in Act 3. | |
| L4 | Self-Evolving World Ecosystem | Autonomous world evolution with emergent behaviors based on L2 and L3, requiring advanced Intelligence module. | E.g. NPCs self-organize governments and trade as population increases. |
Real-Time Interactive Video Generation
Real-Time Interactive Video Generation creates low-latency video streams that respond continuously to user actions, camera motion, text, or audio.
See all 90 papers