Reinforcement learning (RL) is a powerful framework for robotic control, yet its practical application is often hindered by high sample complexity. This is particularly restrictive in physical domains where interaction data is costly. While the world often exhibits geometric and physical symmetries, standard RL algorithms typically fail to exploit this structure. In this paper, we demonstrate that exploiting group symmetries significantly reduces the sample complexity of RL. Focusing on finite-horizon Markov decision processes, we find that leveraging homomorphisms induced by group symmetries significantly reduces the theoretical upper and lower bounds on the number of environment interactions required to reach an optimal return. We further extend these bounds to continuous state and action spaces, providing corresponding sample-complexity guarantees under appropriate regularity assumptions. Beyond theory, we validate our findings through controlled experiments and demonstrate the advantages of symmetry-aware policy learning on high-dimensional continuous robotic simulations. Our results show that integrating symmetry into the learning pipeline yields substantial gains in sample efficiency and performance, offering a principled path toward more data-efficient robotics.
Figures & tables
Figure 1: The homomorphism h maps equivalent states in M to representative states in the quotient MDP Mˉ , reducing the effective sample complexity from O~(kM) to O~(kMˉ) with kMˉ≪kM .
Figure 2: Example where homomorphism h maps the original MDP M to the quotient MDP Mˉ .
Figure 3: Theoretical comparison between unstructured and symmetry-aware sample complexity for a graph-based MDP with ∣S∣=105 states before quotienting by symmetry. For the ring graph, the symmetry group has full state-size symmetry, ∣G∣=105 , yielding a potential five-order-of-magnitude reduction in the number of episodes required to reach 95% optimality with high probability.
Figure 4: Performance across the 1D cyclic ring, 3D cyclic ring, and Rubik’s cube experiments. Symmetry-aware learning reduces the number of episodes needed for better performance.
Table 5
Figure 5: Continuous-control manipulation benchmarks comparing symmetry-aware and non-equivariant learning across multiple random seeds.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Illustrations of the 1D cyclic ring, 3D cyclic ring, and Rubik’s cube experiments.
Figure 7: Rubik’s cube canonicalization. A scrambled state s (left) is replaced by the lexicographically smallest representative of its orbit O(s)={g⋅s:g∈Oh} under the 24 -element rotation group Oh≅S4
Figure 8: Illustrations of the block stacking and nut assembly robotic manipulation benchmarks.
Symmetries play a central role in reducing the complexity of reinforcement learning problems, yet most existing approaches rely on fixed group actions or predefined state abstractions. Classical reinforcement learning algorithms typically assume a globally structured Markov decision process with uniformly applicable actions and transitions, an assumption that limits their ability to exploit modularity and local, context-dependent regularities present in many realistic environments. We propose a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and support the dy- namic discovery of equivalence structures during interaction. The agent maintains orbit representatives together with transporters that map raw states to canonical forms, enabling learning and decision-making to be performed in a symmetry-reduced space while preserving local distinctions. Empirical results demonstrate that the proposed groupoid-based approach improves sample efficiency and convergence in dense and large-scale environments exhibiting strong partial symmetries, yielding substantial performance gains over standard Q-learning. These findings show that dynamically exploiting local symmetry provides a practical and mathematically principled route to scalable and generalisable reinforcement learning.
Ben Opperman, Eduardo Alonso, Esther Mondragón
CitAI, AI Research Centre, City St George’s, University of London, London, United Kingdom
Robots exhibit a rich variety of symmetries arising from their mechanical structure and the properties of their tasks. Although many robotics problems exhibit several symmetries simultaneously, existing approaches typically treat them in isolation, failing to exploit their combined potential. This paper introduces cross-space symmetry compositions, a framework for learning robot policies that are jointly equivariant to multiple symmetries across configuration and task spaces. Leveraging the differential-geometric structure of the forward kinematics map, we both descend symmetries from configuration to task space and lift symmetries from task to configuration space, enabling their composition within a unified representation space. We validate our framework on simulated and real-world experiments on a dual-arm robot, demonstrating that jointly leveraging multiple symmetries yields improved generalization. Video and source code are available at https://sites.google.com/view/cross-space-symmetries.
Reinforcement learning has long struggled with poor sample efficiency. One promising approach to mitigate this problem is leveraging group-invariant Markov Decision Processes (G-invariant MDPs). Existing works in this direction have primarily focused on image-based RL and rotational symmetry such as SO(2), leaving state-based RL and reflection symmetry largely underexplored. In this work, we focus on state-based continuous control tasks and exploit reflection symmetry by introducing Reflex, a paradigm that seamlessly integrates with both on-policy and off-policy RL algorithms. We formalize two types of reflection-axial reflection and bilateral reflection, and characterize their corresponding transformations. Building on a theoretical analysis of symmetry-preserving optimal value functions and policies, Reflex integrates reflection symmetry into policy learning through principled symmetry regularization mechanisms. We integrate Reflex with PPO and SAC, and evaluate it on a suite of OpenAI Gym and DeepMind Control benchmarks, demonstrating superior performance over standard baselines while improving sample efficiency. Our code is available at https://github.com/TonyStark042/Reflex.
Shuai Zhen, Yifan Zhang, Yuling Wang +1
Beijing University of Posts and Telecommunications · Hangzhou Dianzi University