cs.CVOct 14, 2026

RIGen: Beyond Action Following with Reciprocal Interaction in Generative Gameplay

Authors: Mengfei Li, Yi Zhang, Xingtong Ge, Lunjie Zhu, Fangyu Lin, Peng Li, Guanglu Song, Yu Liu, 

Organizations: The Hong Kong University of Science and Technology · Vivix Group Limited

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Generative gameplay requires more than immediate responses to player actions.Actions should have persistent consequences, while the generated world should act according to its own state and rules, shaping the player's situation and subsequent choices. Video generation captures local dynamics, but limited history, partially observed game state, and the need for contextual behavior decisions make such interactions difficult to sustain through audiovisual prediction alone. We introduce \rig (Reciprocally Interactive Generation), a framework for reciprocal interaction in generative gameplay. \rig combines persistent state and configurable rules, contextual interaction reasoning, and action-conditioned audiovisual generation. Together, these components make interaction consequences and actor behavior jointly determine generated content while maintaining consistency between game state and visible outcomes. We construct a Xonotic gameplay dataset to train an audiovisual generator for learning dynamics and interactions. Further post-training enables \rigarena, an interactive generative first-person shooter (FPS) prototype. Experiments demonstrate more reliable combat outcomes and reciprocal interaction between the player and the generated world. Project page: https://murphylmf.github.io/RIGen/.

Figures & tables

Figure 1: Reciprocal interaction in RIGen. Players affect the generated world, whose rules, persistent state, and characters in turn shape gameplay. (a) Attacks have state-dependent consequences. (b) Enemy actions, resource constraints, and respawn rules shape the player’s survival and continued play. (c) Opponents adapt their behavior to their health and current combat conditions.
Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Training stageBackboneAction branches
Basic Dynamics adaptation10−610^{-6}10−510^{-5}
Interaction adaptation5×10−75\times 10^{-7}10−710^{-7}
Arena bidirectional adaptation5×10−75\times 10^{-7}10−710^{-7}
Sliding-window autoregressive training10−610^{-6}10−510^{-5}
Consistency distillation10−610^{-6}10−610^{-6}
Distribution matching10−510^{-5}10−510^{-5}
Appendix
Table 1: Generator training learning rates.
SettingParametersFPS
SCOPE∼\sim 5B4.12
LingBot-World 2.014B4.47
RIGen (Replay)22B4.41
RIGen (Online)22B3.93
Appendix
Table 2: Generation throughput at 832×480832\times 480 . Parameters count the generation backbone. Both RIGen settings include refinement.
More in this topic

Real-Time Interactive Video Generation

Real-Time Interactive Video Generation creates low-latency video streams that respond continuously to user actions, camera motion, text, or audio.

See all 79 papers

Explore similar work

CardsList

  1. BoxWorld: Let World Engines Control and Video Generators Render

    Oct 26, 2026Zike Wu, Ahmed Hatem, Chintan Trivedi +4Real-Time Interactive Video GenerationInteractive World Models

    University of British Columbia · Vector Institute for AI · Huawei Canada +1

  2. StatePlay: State-Aware Game World Models for Mechanics-Consistent Generation

    Jul 29, 2026Zijun Lin, Zeqing Wang, Cheston Tan +2World ModelsAction-Conditioned World Models

    Tencent · Nanyang Technological University · Centre for Frontier AI Research, A*STAR +1

  3. Magpie: Real-Time World Renderer for Interactive Games

    Aug 27, 2026Xiaoyu Zhan, Xinyu Wang, Xiaohong Zhang +6Real-Time Interactive Video Generation

    Mogo AI Ltd. · Nanjing University · University of Southampton