PCGRL for Complex Platformers: Synthesizing Playable and Diverse Levels via Inflated Envelopes and Forward Curricula
Organizations: Harbin Institute of Technology · The University of Hong Kong
Abstract
Procedural Content Generation via Reinforcement Learning (PCGRL) has emerged as a promising paradigm for automated game development. However, existing PCGRL frameworks still suffer from severe exploration bottlenecks and scaling limitations when faced with long-sequence generation tasks governed by strict rigid-body rules and non-linear kinematic boundaries (e.g., platformer constraints). Consequently, traditional approaches frequently fail to maintain reliable level solvability on large-scale canvases. To address these challenges, we present a physics-aware, curriculum-guided PCGRL framework. First, we construct an "Inflated Volume Envelope" mechanism, which provides a physics-constrained representation that incorporates continuous kinematic feasibility into the discrete state space of the MDP. Second, to alleviate the sparse reward challenge induced by extremely long decision sequences, we introduce intermediate state-transition feedback and a multi-stage forward extension curriculum learning framework, enabling informative value estimation and stable credit assignment over long-horizon trajectories. Finally, to resolve the issue where deterministic policies after convergence generate highly monotonous map paths with a lack of diversity, we activate high-entropy stochastic inference during the synthesis phase, regularized by a post-hoc parallel rejection filtering mechanism. Experimental results demonstrate that our framework significantly enhances structural diversity while maintaining high physical playability across multi-scale environments. Comparative evaluations across multi-scale canvases demonstrate that our framework maintains superior convergence stability and dimensional scalability, reliably synthesizing solvable levels where traditional PCGRL baselines degrade.
Figures & tables
| Parameter / Notation | Value / Setting |
|---|---|
| Avatar Kinematics & Simulator | |
| Gravitational acceleration ( ) | |
| Running horizontal speed ( ) | |
| Jump vertical impulse ( ) | |
| Terminal falling velocity ( ) | |
| Avatar collision footprint ( ) | |
Procedural Content Generation
Procedural content generation creates game levels, 3D worlds, assets, stories, and simulation environments through algorithms and learned models.
See all 73 papers