Modular Reinforcement Learning

Modular reinforcement learning (MRL) aims to improve the efficiency and generalizability of reinforcement learning by decomposing complex tasks into smaller, independently learned sub-tasks managed by specialized modules. Current research focuses on developing architectures that effectively integrate diverse knowledge sources (e.g., rules, sub-goals) and handle heterogeneous data representations within these modular systems, often employing deep learning techniques for policy learning and module coordination. This approach shows promise in tackling challenging control problems in robotics, manufacturing, and even automated theorem proving, offering advantages in sample efficiency, robustness, and the ability to scale to high-dimensional action spaces.

Papers

April 1, 2024

CAMO: Correlation-Aware Mask Optimization with Modulated Reinforcement Learning
Xiaoxiao Liang, Haoyu Yang, Kang Liu, Bei Yu, Yuzhe Ma
Prey Camouflage Mask Optimization Optical Proximity Correction Modular Reinforcement Learning

June 1, 2023

Heterogeneous Knowledge for Augmented Modular Reinforcement Learning
Lorenz Wolf, Mirco Musolesi
Dynamic ModulE Heterogeneous Knowledge Modular Reinforcement Learning

March 8, 2023

Dextrous Tactile In-Hand Manipulation Using a Modular Reinforcement Learning Architecture
Johannes Pitz, Lennart Röstel, Leon Sievers, Berthold Bäuml
Robotic Hand Differentiable Particle Filter Real Robotic System Modular Reinforcement Learning

October 26, 2022

Low-Rank Modular Reinforcement Learning via Muscle Synergy
Heng Dong, Tonghan Wang, Jiayuan Liu, Chongjie Zhang
Robot Morphology Muscle Synergy Synergy Oriented LeARning Multi Morphology Controller Modular Reinforcement Learning

September 6, 2022

Project proposal: A modular reinforcement learning based automated theorem prover
Boris Shminke
Reinforcement Learning Theorem Proving State Representation Proposal Balance Refinement Modular Reinforcement Learning

April 13, 2022

Modularity benefits reinforcement learning agents with competing homeostatic drives
Zack Dulberg, Rachit Dubey, Isabel M. Berwian, Jonathan D. Cohen
Reinforcement Learning Agent Smith Agent System Q$ Learning Homeostatic Design Confers Scalar Reward Modular Reinforcement Learning