Reinforcement Learning

Reinforcement learning (RL) focuses on training agents to make optimal decisions in an environment by learning through trial and error, aiming to maximize cumulative rewards. Current research emphasizes improving RL's efficiency and robustness, particularly in areas like human-in-the-loop training (e.g., using human feedback to refine models), handling uncertainty and sparse rewards, and scaling to complex tasks (e.g., robotics, autonomous driving). Prominent approaches involve various policy gradient methods, Monte Carlo Tree Search, and the integration of large language models for improved decision-making and task decomposition. These advancements are driving progress in diverse fields, including robotics, game playing, and the development of more human-aligned AI systems.

Papers

October 22, 2024

Uncovering RL Integration in SSL Loss: Objective-Specific Implications for Data-Efficient RL
Ömer Veysel Çağatan, Barış Akgün
Reinforcement Learning Data Efficient Replay Based SSL Algorithm DeepMind Control Suite Domain Loss
Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models
Muhan Lin, Shuyang Shi, Yue Guo, Behdad Chalaki, Vaishnav Tadiparthi, Ehsan Moradi Pari, Simon Stepputtis, Joseph Campbell, Katia Sycara
Language Model Reinforcement Learning Reward Function Reward Model Noisy Feedback
Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning
Dongsu Lee, Minhae Kwon
Reinforcement Learning LeArning Abstract Multi Agent Interaction Multi User Episodic Future Thinking
Reinforcement learning on structure-conditioned categorical diffusion for protein inverse folding
Yasha Ektefaie, Olivia Viessmann, Siddharth Narayanan, Drew Dresser, J. Mark Kim, Armen Mkrtchyan
Reinforcement Learning Protein Design Complex Structure Inverse Folding
Reinforcement Learning for Data-Driven Workflows in Radio Interferometry. I. Principal Demonstration in Calibration
Brian M. Kirk, Urvashi Rau, Ramyaa Ramyaa
Reinforcement Learning Calibration Performance Radio Source Observational Study Interferometric Imaging
Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards
Alexander G. Padula, Dennis J.N.J. Soemers
Reinforcement Learning Reward Function Proximal Policy Optimization LLM Training Passive Learning
Science Out of Its Ivory Tower: Improving Accessibility with Reinforcement Learning
Haining Wang, Jason Clark, Hannah McKelvey, Leila Sterman, Zheng Gao, Zuoyu Tian, Sandra Kübler, Xiaozhong Liu
Language Model Reinforcement Learning Science Journalism Accessibility Issue Language Reward Readability Metric Dark Jargon Scientific Abstract
Optimal Design for Reward Modeling in RLHF
Antoine Scheid, Etienne Boursier, Alain Durmus, Michael I. Jordan, Pierre Ménard, Eric Moulines, Michal Valko
Reinforcement Learning Reinforcement Learning From Human Feedback Optimal Design Pairwise Preference Bandit Learning Reward Model Training
Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shifts
Sheryl Paul, Jyotirmoy V. Deshmukh
Reinforcement Learning Robust Policy Prior Policy Policy Adaptation Deep Survival Environmental Change
Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning
Zongmeng Zhang, Yufeng Shi, Jinhua Zhu, Wengang Zhou, Xiang Qi, Peng Zhang, Houqiang Li
Large Language Model Language Model Reinforcement Learning Retrieval Augmentation Retrieval Augmented Language Model
Guiding Reinforcement Learning with Incomplete System Dynamics
Shuyuan Wang, Jingliang Duan, Nathan P. Lawrence, Philip D. Loewen, Michael G. Forbes, R. Bhushan Gopaluni, Lixian Zhang
Reinforcement Learning Model Free Reward Function Design System Dynamic
Sample-Efficient Curriculum Reinforcement Learning for Complex Reward Functions
Kilian Freitag, Kristian Ceder, Rita Laezza, Knut Åkesson, Morteza Haghir Chehreghani
Reinforcement Learning Reward Function Simple Function Curriculum Reinforcement Learning Scalar Reward
LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"
Som Sagar, Aditya Taparia, Ransalu Senanayake
Reinforcement Learning Neural Network Diffusion Model Red Teaming Leg Failure Partial Feedback Failure Transition
Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment
Mingzhi Wang, Chengdong Ma, Qizhi Chen, Linjian Meng, Yang Han, Jiancong Xiao, Zhaowei Zhang, Jing Huo, Weijie J. Su, Yaodong Yang
Reinforcement Learning Preference Optimization Mirror Descent Self Play Two Player Zero Sum Language Model Alignment Last Iterate Convergence
QuasiNav: Asymmetric Cost-Aware Navigation Planning with Constrained Quasimetric Reinforcement Learning
Jumman Hossain, Abu-Zaher Faridee, Derrik Asher, Jade Freeman, Theron Trout, Timothy Gregory, Nirmalya Roy
Reinforcement Learning Autonomous Navigation Participation Constraint Efficient Navigation Traversability Analysis Quasimetric Learning
Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Games
Kimiya Saadat, Richard Zhao
Reinforcement Learning Empirical Study Video Game Self Play Two Player ATARI Game
Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies
Guilherme Christmann, Ying-Sheng Luo, Hanjaya Mandala, Wei-Chao Chen
Reinforcement Learning Deep Reinforcement Learning Reinforcement Learning Environment Regularization Loss Non Converging Artificial Oscillation Continuous Control Policy

October 21, 2024

Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment
Joshua T. S. Hewson
Reinforcement Learning Artificial Intelligence Alignment Problem Human Mind Intrinsic Motivation
RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space
Jingdi Chen, Hanhan Zhou, Yongsheng Mei, Carlee Joe-Wong, Gina Adam, Nathaniel D. Bastian, Tian Lan
Reinforcement Learning Deep Reinforcement Learning Decision Tree Non Euclidean Metric Space Decentralized Multi Agent
Integrating Reinforcement Learning with Foundation Models for Autonomous Robotics: Methods and Perspectives
Angelo Moroncelli, Vishal Soni, Asad Ali Shahid, Marco Maccarini, Marco Forgione, Dario Piga, Blerina Spahiu, Loris Roveda
Reinforcement Learning Foundation Model Robotics Domain NCD Method Legal Autonomy

Reinforcement Learning

Papers

Uncovering RL Integration in SSL Loss: Objective-Specific Implications for Data-Efficient RL

Navigating Noisy Feedback: Enhancing Reinforcement Learning with Error-Prone Language Models

Episodic Future Thinking Mechanism for Multi-agent Reinforcement Learning

Reinforcement learning on structure-conditioned categorical diffusion for protein inverse folding

Reinforcement Learning for Data-Driven Workflows in Radio Interferometry. I. Principal Demonstration in Calibration

Exploring RL-based LLM Training for Formal Language Tasks with Programmed Rewards

Science Out of Its Ivory Tower: Improving Accessibility with Reinforcement Learning

Optimal Design for Reward Modeling in RLHF

Survival of the Fittest: Evolutionary Adaptation of Policies for Environmental Shifts

Trustworthy Alignment of Retrieval-Augmented Large Language Models via Reinforcement Learning

Guiding Reinforcement Learning with Incomplete System Dynamics

Sample-Efficient Curriculum Reinforcement Learning for Complex Reward Functions

LLM-Assisted Red Teaming of Diffusion Models through "Failures Are Fated, But Can Be Faded"

Magnetic Preference Optimization: Achieving Last-iterate Convergence for Language Model Alignment

QuasiNav: Asymmetric Cost-Aware Navigation Planning with Constrained Quasimetric Reinforcement Learning

Enhancing Two-Player Performance Through Single-Player Knowledge Transfer: An Empirical Study on Atari 2600 Games

Benchmarking Smoothness and Reducing High-Frequency Oscillations in Continuous Control Policies

Combining Theory of Mind and Kindness for Self-Supervised Human-AI Alignment

RGMDT: Return-Gap-Minimizing Decision Tree Extraction in Non-Euclidean Metric Space

Integrating Reinforcement Learning with Foundation Models for Autonomous Robotics: Methods and Perspectives