quant-phNov 7, 2026

Trainability-Simulability Tradeoff in Parameterized IQP-based Reinforcement Learning Policies

Authors: André Sequeira

Organizations: Quantum Software Engineering, HASLab INESC TEC, Braga, Portugal

Abstract

Parameterized instantaneous quantum polynomial-time (IQP) circuits generate distributions conjectured to be hard to sample classically. As reinforcement learning (RL) policies, their 2n2^n measurement outcomes are typically mapped to a smaller action space by a classical decoder. This aggregation can mitigate initialization concentration while making the policy classically simulable. We characterize this tradeoff through the decoder's Fourier--Pauli expansion. Under independent uniform initialization, we express policy variance as an exact sum over groups of nonconstant Pauli components with identical active-generator signatures. Each group contributes its squared aggregate coefficient times 2−r2^{-r}, where rr is the binary rank of its active generators. An inverse-polynomial aggregate coefficient with logarithmic rank prevents exponential policy concentration. Sparse Pauli expansions and contiguous action bins permit efficient classical additive estimation of action probabilities. We introduce a majority decoder combining protected low-weight components with exponentially large exact Fourier support, preserving initialization nonconcentration while excluding polynomial-size enumeration of the exact expansion. Numerical experiments corroborate the variance laws. In ten-qubit CartPole experiments, majority achieves higher returns than recursive parity and the tested local readouts, while higher-weight Pauli sectors contribute an increasing relative share of gradient magnitude during training. A complementary 21-qubit block-chain policy outperforms an extensive pool of fully connected classical neural networks in episode efficiency, even though its bounded-width structure permits efficient classical simulation. These results guide decoder design for quantum and quantum-inspired RL, distinguishing policy nonconcentration, learning performance, and classical simulability.

Figures & tables

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Reinforcement Learning for Parameterized Quantum State Preparation: A Comparative Study

    Feb 18, 2026Gerhard Stenzel, Isabella Debelic, Michael Kölle +4Quantum Circuit SynthesisQuantum State Preparation

  2. Towards Surrogate Based Dequantization of Quantum Reinforcement Learning

    Sep 14, 2026Pablo Rodriguez-Grasa, Sofiene Jerbi, Mikel Sanz +1Reinforcement Learningfitted q-iteration

  3. QnRL: Quantum-Native Reinforcement Learning

    Jun 6, 2026Alexander DeRieux, Walid SaadDistributional RLQuantum Generative Modeling