cs.LGNov 11, 2026

Latent Imagination for Parameterized Action Spaces

Authors: Atahan Cilan, İhsan Mert Muhacıroğlu, Ümit Can Bekar

Organizations: Turkish Aerospace.

Abstract

Parameterized-action MDPs (PAMDPs) ask an agent to choose a discrete action together with continuous parameters whose meaning depends on that choice. Model-free PAMDP methods act fast but need many interactions; model-based methods are sample-efficient but plan at every decision, which makes them slow to act. We introduce HyLI (Hybrid Latent Imagination), a model-based agent that learns a hybrid policy entirely in the latent imagination of a world model and acts with a single forward pass. To address learning problems that the PAMDP structure creates, HyLI adds counterfactual mode evaluation, which gives every discrete action a learning signal at every imagined state, and success-window replay, which keeps rare successes in training. On seven standard PAMDP benchmarks, HyLI is within one published standard deviation of, or above, the published score of the strongest model-free baseline in our comparison on all 7 tasks, reaches 90% of that score with 2.4x fewer environment steps than that baseline (5.0x when its reward-free pre-training is counted), and decides 67x faster than the model-based planner DLPA at its published planner settings. It is the only method in our comparison that is both more sample-efficient than that baseline and fast enough for real-time control.

Figures & tables

Appendix figures & tables22 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Mind Dreamer: Untethering Imagination via Active Causal Intervention on Latent Manifolds

    May 15, 2026Shaojun Xu, Xiaoling Zhou, Yihan Lin +4Reinforcement LearningCredit Assignment in RL

  2. Latent Action Reparameterization for Efficient Agent Inference

    May 18, 2026Wenhao Huang, Qingwen Zeng, Qiyue Chen +11LLM Inference EfficiencyLatent Action Learning

  3. IMWM: Intuition Models Complement World Models for Latent Planning

    Jun 1, 2026Baoqi Gao, Ruize Han, Miao Wang +1World Model-Based PlanningGoal-Conditioned Planning