Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits

Yu-Ting Lee; Samuel Yen-Chi Chen; Fu-Chieh Chang

cs.LGMay 5, 2026

Quantum Hierarchical Reinforcement Learning via Variational Quantum Circuits

Authors: Yu-Ting Lee, Samuel Yen-Chi Chen, Fu-Chieh Chang

Organizations: Graduate Institute of Communication Engineering, National Taiwan University, Taipei, Taiwan · Wells Fargo, New York, NY, USA · MediaTek Inc., Hsinchu, Taiwan

Abstract

While parameterized quantum computations have shown success in standard reinforcement learning (RL), whether these advantages adapt to hierarchical RL (HRL) remains a critical open question. This work demonstrates that variational quantum circuits (VQCs) can effectively enhance HRL agents based on the option-critic architecture. Evaluated in standard environments, a hybrid HRL agent with a quantum feature extractor outperforms classical baselines while using fewer parameters. We also identify an architectural bottleneck: using VQCs for option-value estimation severely degrades learning. Further ablations reveal how quantum circuit design affects performance. Our work establishes design principles for parameter-efficient hybrid HRL agents.

Explore similar work

QnRL: Quantum-Native Reinforcement Learning
Jun 6, 2026Alexander DeRieux, Walid SaadQ-Learning
Towards Surrogate Based Dequantization of Quantum Reinforcement Learning
Sep 14, 2026Pablo Rodriguez-Grasa, Sofiene Jerbi, Mikel Sanz +1Q-Learning Quantizer
Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation
Jun 29, 2026Javier Lazaro, Juan-Ignacio Vazquez, Pablo Garcia-BringasVariational Quantum Circuits Knowledge Distillation

Jun 6, 2026 · quant-phJ/K move · Enter open · S save

QnRL: Quantum-Native Reinforcement Learning

Alexander DeRieux, Walid Saad

Bradley Department of Electrical and Computer Engineering, Virginia Tech Institute for Advanced Computing, Alexandria, VA 22305 USA

Quantum reinforcement learning (QRL) is a promising approach to learn effective decision strategies across several applications with stochastic environments. Instead of directly modeling the random variables that govern these environments, existing QRL architectures indirectly approximate environment behavior by estimating expected outcomes, which limits their expressive power and adaptive potential. Overcoming such challenges requires a novel QRL approach that exploits the distributional nature of quantum computers to directly model environment random variables as quantum state distributions. Hence, in this paper, a novel framework dubbed quantum-native reinforcement learning (QnRL) is proposed. QnRL is a distributional RL framework that learns conditional distributions naturally in Hilbert space via superimposed and entangled quantum states. Thus, QnRL can directly model the behavior of stochastic learning environments via the natural properties of quantum systems. QnRL accomplishes this via a novel, proposed quantum amplitude kickback (QuAK) algorithm that enables comparing the

n

-th power of the

m

-th moment of multiple superimposed distributions. It is theoretically proven that a conditional action policy distribution is distilled from the moments of a quantum generative model entirely within Hilbert space via QuAK, and optimized via QnRL. This complex distribution composition is also shown to provide extra dimensions for expressing environment correlations that are unknown to purely classical and classically-sampled quantum distributional models. Experimental results across diverse environments show that QnRL achieves up to

82.9\%

higher evaluation scores, with up to

94.3\%

fewer parameters on average, more accurately estimates the expected return for unseen observations, and better adapts to varying stochastic conditions compared to the baseline.

Q-Learning

Open paper Ask about it PDF

Abstract

Explore similar work

QnRL: Quantum-Native Reinforcement Learning

Towards Surrogate Based Dequantization of Quantum Reinforcement Learning

Staged Hybridisation for Visual Quantum Reinforcement Learning via Knowledge Distillation