LLM-Guided RL

RL: Reinforcement Learning

This field studies how large language models can help reinforcement-learning agents learn useful behavior, and how reinforcement learning can improve language models’ reasoning and tool use. Common approaches use models to propose or refine rewards, provide selective hints or curricula, generate plans and reusable skills, or supply semantic knowledge that is distilled into policies. Work spans sparse-reward robotics and game environments, multi-agent coordination, autonomous driving, and language-model reasoning tasks. A central challenge is making guidance reliable and transferable: changing or inaccurate rewards can destabilize learning, and gains against model-based judges may reflect reward hacking rather than better real-world behavior.

Latest papers

140 papers19 papers in the last four weeks, up 46% on the four weeks before. 0.1% of all new papers.

$R^2E:$ A Role-driven Reward Evolutionary Framework for Automated Reward Function Design
Oct 23, 2026Shouhao Chang, Xuan Liu, Hongye Zhu +2Reward Design for RL LLM-Guided RL
Hunan University · Central South University
MOER: Mix-Policy Optimization under Expert Prefix Guidance with Selective Rollback
Oct 22, 2026Xuyang Zhong, Jiuding Yang, Yiwen Guo +1RL for Language Model Reasoning Policy Optimization
City University of Hong Kong · LIGHTSPEED · Independent Researcher
RANGER: Retrieval Agent for Navigating Graphs with Semantic Evaluation Rewards
Oct 20, 2026Artem Mikhaylov, Pavel Konoplev, Nikolay Efanov +2Knowledge Graph Question Answering LLM-Guided RL
Skolkovo Institute of Science and Technology (Skoltech), Moscow, Russia · Moscow Institute of Physics and Technology (MIPT), Moscow, Russia · AIRI, Moscow, Russia
HarnessRL: Co-Evolving Policies with a Just-in-Time Training Harness for Open-Ended RL
Oct 19, 2026Xiaoqiang Wang, Ting-En Lin, Yuchuan Wu +7Rubric-Based RL Reward Design for RL
Alibaba Token Hub, Alibaba Group · Université de Montréal · Mila - Quebec AI Institute
ClinSDT: LLM-Encoded Clinical Semantic Guidance for Sepsis Treatment via Decision Transformer
Oct 18, 2026Yanyan Wu, Guanyu Zhu, Changbin Zhao +7Healthcare Clinical Decision-Making
South China Agricultural University, Guangzhou, China · The Ohio State University, Columbus, Ohio, United States
Progress Reward Model for Reinforcement Learning via Large Language Models
Oct 15, 2026Xiuhui Zhang, Ning Gao, Xingyu Jiang +4Reward Design for RL LLM-Guided RL
Beihang University, 37 Xueyuan Road, Haidian District, Beijing
Who Sets the Pace of Judge Hacking? Judge Capability versus the Policy-Judge Gap
Oct 11, 2026Haosen WangReward Hacking LLM-Guided RL
Beyond Temporal Proximity: Explanatory World Models for Credit Assignment in Reinforcement Learning
Oct 9, 2026Oriol Corcoll, Qiyao Wei, Ciarán Mark Gilligan-LeeRL for Recommendation Step-Level Credit Assignment in RL
Spotify · University of Cambridge
EliteTable: Self-Improving Program Search for Executable Control Policies
Oct 8, 2026Stuart FarmerProgram Synthesis LLM-Based Program Synthesis
Casa Dilauro Independent Research (casadilauro.com)
Can Jev be Your Q or Policy in Reinforcement Learning?
Oct 8, 2026Yi Ma, Tianpei Yang, Yaodong Yang +2LLM-Guided RL Sample-Efficient RL
Shanxi University · Nanjing University · The Chinese University of Hong Kong +2
Ask the Expert: LLM-Guided Reinforcement Learning for Autonomous Cyber Defense
Oct 7, 2026Fernando Martinez, Abhishek Satyam, Tao Li +3Autonomous Cyber Defense LLM-Guided RL
Department of Computer and Information Sciences, Fordham University, New York, USA · Department of Systems Engineering, Stevens Institute of Technology, New Jersey, USA
Rationale-Guided Policy Optimization: Learning to Reason with Adaptive Rationale Scaffolding
Oct 5, 2026Hoang Phan, Minh Pham, Chau Pham +3RL for Language Model Reasoning LLM-Guided RL
New York University · University at Buffalo · Monash University
An LLM-in-the-loop RL Framework for Bioinformatics Feature Selection
Oct 4, 2026Xinyuan Wang, Deepti Agrawal, Yanjie FuReinforcement Learning LLM-Guided RL
School of Computing and AI Arizona State University Tempe, USA
Anticipating the Consequences of Curriculum Decisions with Large Language Models
Oct 3, 2026Octavio Pappalardo, Nathan Herr, Tim RocktäschelCurriculum Learning Curriculum RL
University College London
LocusRL: Diagnosing LLM Reward and Policy Interventions in Competitive Games
Oct 3, 2026Chengyu Luan, Bo Xin, Songyan Guo +4RL for Game AI Reinforcement Learning
Harbin Institute of Technology
A Bird's-Eye View of Iterative Reward Design
Oct 3, 2026Logan Mondal Bhamidipaty, Lauren Robson, Linda Petrini +2Reward Design for RL LLM-Guided RL
Anthropic Fellows Program · University of Edinburgh · Anthropic
ROOT: Discovering Rewards for User-Specified Embodied Behaviors
Oct 3, 2026Eren Sadikoglu, Aditya Taparia, Xinyuan Liu +1Reward Design for RL Robotic RL
Robotics and Autonomous Systems, Arizona State University, Tempe, Arizona, USA · School of Computing and Augmented Intelligence, Arizona State University, Tempe, Arizona, USA
Iterative Policy Refinement through Semantic Rollout Analysis
Oct 1, 2026Feiyu Gavin Zhu, Qi Xu, Zhifei Deng +4Policy Optimization LLM-Guided RL
Carnegie Mellon University · Meta
Guide, Then Let Go: Gap-Adaptive Teacher Scheduling for Sparse-Reward Agentic RL
Sep 29, 2026Youling Huang, Tiankuo Xu, Jiaji Liu +10Agentic RL LLM-Guided RL
DUT · Kuaishou · XJTU +5
Expected Reasoning-Step Return Unifies On-Policy Learning from Rewards and Teachers
Sep 26, 2026Qiangqiang He, Jin LiRL for Language Model Reasoning LLM-Guided RL
State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · College of Software Engineering, Southeast University, Nanjing, China
Privacy-Preserving Prompted Policy Search for Robotic Control
Sep 24, 2026Ali Irshayyid, Feng Lin, Chong Li +1LLM-Guided RL Robot Policy Learning
Department of Electrical and Computer Engineering, Oakland University, Rochester, MI 48309 USA · Department of Electrical and Computer Engineering, Wayne State University, Detroit, MI 48202, USA · OORT and the Department of Electrical Engineering, Columbia University, NYC, 10027, USA
SynthDemo-RL: Breaking the Zero-Reward Barrier in VLA Adaptation with LLM-Guided Synthetic Demonstrations
Sep 18, 2026Hiroaki Kingetsu, Hiroaki Kurihara, Kaoru Yokoo +2Vision-Language-Action Models RL Fine-Tuning
Fujitsu Limited, Kawasaki, Japan · The Institute of Statistical Mathematics, Tokyo, Japan
Agentic AI Networking for Heterogeneous Unmanned Aerial Systems in Low-Altitude Wireless Networks
Sep 17, 2026Nguyen Duc Minh Quang, Chang Liu, Shuangyang Li +1LLM-Guided RL Wireless Communications
School of Computing, Engineering, and Mathematical Sciences, La Trobe University, Melbourne, VIC, Australia · Chair of Communications and Information Theory, Technical University of Berlin, 10623 Berlin, Germany · School of Electrical Engineering and Telecommunications, University of New South Wales, Sydney, NSW, Australia
M $^3$ P-R1: Reinforcement Learning for Large Language Model Guided Multi-Modal Motion Planning via MIP Code Generation
Sep 16, 2026Xingpeng Sun, Zherong Pan, Kai Cheng +3Constrained Motion Planning Large Language Model-Based Robot Planning
Purdue University United States · LightSpeed Studios United States
Specifying Reward Functions for RL Without Environment Sampling
Sep 14, 2026Stephane Hatgis-Kessell, W. Bradley Knox, Emma BrunskillLLM-Guided RL Reward Design for RL
1Computer Science Department, Stanford University · 2Computer Science Department, The University of Texas at Austin
Unlocking the Unsolvable: Teacher-Guided Curriculum for Data-Efficient RLVR
Sep 12, 2026Yukang Zhu, Zhen HanRL for Language Model Reasoning Curriculum RL
Independent Researcher · Amazon
LLM-Enhanced Multi-Agent Reinforcement Learning for Unified Electric Vehicles-Charging Station-Grid Optimization in Public Charging Systems
Sep 12, 2026Yang Zhang, Lindong Xie, Chongyu Wang +3Electric Vehicle Charging Multi-Objective Reinforcement Learning
Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Galvin Center for Electricity Innovation, Illinois Institute of Technology, Chicago, IL 60616, USA
CanvasAnneal: Curriculum Reinforcement Learning for Diffusion Language Models
Sep 11, 2026Blake Olson, Yuhang Song, Emmett McQuinn +1RL for Language Model Reasoning Curriculum RL
Google DeepMind
Smart Adaptive Computing Across the Continuum: LLMs in IoT-Edge-Cloud Resource Management
Sep 8, 2026Antonino Vaccarella, Lanpei Li, Vincenzo Lomonaco +1Language Model-Based Control LLM-Guided RL
Institute of Information Science and Technologies “Alessandro Faedo” (ISTI), National Research Council of Italy (CNR), 56124 Pisa, Italy · Department of Computer Science, University of Pisa, 56127 Pisa, Italy · Department of AI, Data and Decision Sciences, LUISS University, 00197 Rome, Italy
From LLM-Generated Specifications to Learned Quadruped Locomotion
Sep 7, 2026Merve Atasever, Keyan Azbijari, Cagan Bakirci +5LLM-Guided RL Quadruped Locomotion
Department of Computer Science University of Southern California Los Angeles, CA, USA · University of Florida Gainesville, FL, USA

Oct 23, 2026J/K move · Enter open · S save

$R^2E:$ A Role-driven Reward Evolutionary Framework for Automated Reward Function Design

Shouhao Chang, Xuan Liu, Hongye Zhu, Xinning Chen+1

Hunan University · Central South University

Reward Design for RL LLM-Guided RL Evolutionary RL Robotic RL

Open paper AskPDF arXiv

Designing effective reward functions is a critical challenge in Reinforcement Learning (RL), which traditionally requires costly manual trial-and-error. While Large Language Model (LLM)-based methods have shown promise in reward design, undifferentiated sampling and limited use of historical feedback often lead to insufficient exploration of the reward space and optimization instability. To address these challenges, we propose

R^2E

, a Role-driven Reward Evolutionary framework for automated reward function design that structures the process through explicit role specialization.

R^2E

employs multiple LLM roles with complementary objectives: an Explorer that promotes novelty to expand the search space, a Guardian that performs conservative refinements to improve training stability, and an Evolver that recombines high-performing rewards via crossover and mutation to integrate effective structures. A dynamic scheduling strategy coordinates these roles, progressively shifting the search from broad exploration to focused exploitation. To mitigate optimization instability and provide a medium for cross-role collaboration,

R^2E

incorporates a Global Elite Pool that retains the best-performing rewards to guide subsequent generations, thereby enabling coordinated interaction among roles and ensuring a reliable and consistent refinement process. Extensive experiments on multiple robotic tasks demonstrate the effectiveness of our method.