Programmatic Reinforcement Learning

Programmatic reinforcement learning (PRL) aims to represent reinforcement learning policies as executable programs, enhancing interpretability and generalizability compared to traditional black-box neural network approaches. Current research focuses on improving the efficiency of program search, often leveraging large language models to guide the process and incorporating hierarchical structures like state machines to handle long-horizon tasks. This approach holds significant promise for addressing challenges in cooperative AI, robot control, and other domains requiring explainable and robust AI agents, particularly where human-agent interaction or adaptation to novel environments is crucial.

Papers

October 29, 2024

Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution
Senne Deproost, Denis Steckelmacher, Ann Nowé
Deep Reinforcement Learning Black Box Reinforcement Learning Agent Critic Model Actor Loss Programmatic Reinforcement Learning RL Algorithm

August 8, 2024

KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordination
Yin Gu, Qi Liu, Zhi Li, Kai Zhang
Deep Reinforcement Learning Zero Shot Coordination Cooperative AI Programmatic Reinforcement Learning

May 26, 2024

Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search
Max Liu, Chan-Hung Yu, Wei-Hsu Lee, Cheng-Wei Hung, Yen-Chun Chen, Shao-Hua Sun
Search Space LLM Based Framework Programmatic Reinforcement Learning

February 18, 2024

Theoretical foundations for programmatic reinforcement learning
Guruprerana Shabadi, Nathanaël Fijalkow, Théo Matricon
Reinforcement Learning Optimal Policy Theoretical Foundation Programmatic Reinforcement Learning Programmatic Policy

November 27, 2023

Program Machine Policy: Addressing Long-Horizon Tasks by Integrating Program Synthesis and State Machines
Yu-An Lin, Chen-Tao Lee, Guan-Ting Liu, Pu-Jen Cheng, Shao-Hua Sun
Reinforcement Learning Deep Reinforcement Learning Program Synthesis Long Horizon Task Deep RL State Machine Programmatic Reinforcement Learning Programmatic Policy

October 17, 2023

Learning a Hierarchical Planner from Humans in Multiple Generations
Leonardo Hernandez Cano, Yewen Pu, Robert D. Hawkins, Josh Tenenbaum, Armando Solar-Lezama
Task Planning Real Human Programming Assistance Effective Baseline Complex Program Programmatic Reinforcement Learning Multiple Generation

May 4, 2023

Programming-by-Demonstration for Long-Horizon Robot Tasks
Noah Patton, Kia Rahmani, Meghana Missula, Joydeep Biswas, Işil Dillig
Long Horizon Learning From Demonstration Control Flow Programmatic Reinforcement Learning Programming by Example

January 30, 2023

Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs
Guan-Ting Liu, En-Pei Hu, Pu-Jen Cheng, Hung-yi Lee, Shao-Hua Sun
Reinforcement Learning Problem Solving Policy Alignment Programmatic Reinforcement Learning

June 27, 2022

Programmatic Concept Learning for Human Motion Description and Synthesis
Sumith Kulal, Jiayuan Mao, Alex Aiken, Jiajun Wu
Critical Synthesis Action Sequence Motion Dynamic Hierarchical Motion Programmatic Reinforcement Learning Motion Based Interaction

December 4, 2021

Deep Policy Iteration with Integer Programming for Inventory Management
Pavithra Harsha, Ashish Jagmohan, Jayant R. Kalagnanam, Brian Quanz, Divya Singhvi
Reinforcement Learning Policy Iteration Integer Programming Inventory Management Programmatic Reinforcement Learning Replenishment Decision

Programmatic Reinforcement Learning

Papers

Human-Readable Programs as Actors of Reinforcement Learning Agents Using Critic-Moderated Evolution

KnowPC: Knowledge-Driven Programmatic Reinforcement Learning for Zero-shot Coordination

Synthesizing Programmatic Reinforcement Learning Policies with Large Language Model Guided Search

Theoretical foundations for programmatic reinforcement learning

Program Machine Policy: Addressing Long-Horizon Tasks by Integrating Program Synthesis and State Machines

Learning a Hierarchical Planner from Humans in Multiple Generations

Programming-by-Demonstration for Long-Horizon Robot Tasks

Hierarchical Programmatic Reinforcement Learning via Learning to Compose Programs

Programmatic Concept Learning for Human Motion Description and Synthesis

Deep Policy Iteration with Integer Programming for Inventory Management