LLM-Guided RL

RL: Reinforcement Learning

This field studies how large language models can help reinforcement-learning agents learn useful behavior, and how reinforcement learning can improve language models’ reasoning and tool use. Common approaches use models to propose or refine rewards, provide selective hints or curricula, generate plans and reusable skills, or supply semantic knowledge that is distilled into policies. Work spans sparse-reward robotics and game environments, multi-agent coordination, autonomous driving, and language-model reasoning tasks. A central challenge is making guidance reliable and transferable: changing or inaccurate rewards can destabilize learning, and gains against model-based judges may reflect reward hacking rather than better real-world behavior.

Latest papers

140 papers19 papers in the last four weeks, up 46% on the four weeks before. 0.1% of all new papers.

Discovering Reinforcement Learning Interfaces with Large Language Models
May 5, 2026Akshat Singh Jaswal, Ashish Baghel, Paras ChopraReinforcement Learning LLM-Guided RL
Lossfunk
ARGUS: Policy-Adaptive Ad Governance via Evolving Reinforcement with Adversarial Umpiring
May 4, 2026Deyi Ji, Junyu Lu, Xuanyi Liu +7Non-Stationary RL Online Advertising
Tencent · Dalian University of Technology · Peking University +2
Model-Based Proactive Cost Generation for Learning Safe Policies Offline with Limited Violation Data
May 2, 2026Ruiqi Xue, Lei Yuan, Kainuo Cheng +2LLM-Guided RL Offline RL
National Key Laboratory for Novel Software Technology, Nanjing University · School of Artificial Intelligence, Nanjing University · Polixir Technologies +2
PPO guided Agentic Pipeline for Adaptive Prompt Selection and Test Case Generation
May 1, 2026Gourisetty Venkata Sai Koushik, Dama Aditya, Mahankali Harish Sai +3Automated Software Testing LLM-Guided RL
Department of Computer Science and Engineering, SRM University AP
RHyVE: Competence-Aware Verification and Phase-Aware Deployment for LLM-Generated Reward Hypotheses
Apr 30, 2026Feiyu Wu, Xu Zheng, Zhuocheng Wang +2Reinforcement Learning LLM-Guided RL
School of Cyber Engineering, Xidian University
Bridging Reasoning and Action: Hybrid LLM-RL Framework for Efficient Cross-Domain Task-Oriented Dialogue
Apr 25, 2026Yangyang Zhao, Linfan Dai, Li Cai +2Constrained RL LLM-Guided RL
School of Computer Science and Technology, Changsha University of Science and Technology · School of Computer Science and Engineering, Central South University · Text Computing and Cognitive Intelligence MOE Engineering Research Center, Guizhou University +2
Self-Guided Plan Extraction for Instruction-Following Tasks with Goal-Conditional Reinforcement Learning
Apr 22, 2026Zoya Volovikova, Nikita Sorokin, Dmitriy Lukashevskiy +2Reinforcement Learning LLM-Guided RL
1AXXX · 2MIRAI
Scaling Self-Play with Self-Guidance
Apr 22, 2026Luke Bailey, Kaiyue Wen, Kefan Dong +2Self-Play RL Automated Theorem Proving
Stanford University
Do LLM-derived graph priors improve multi-agent coordination?
Apr 19, 2026Nikunj Gupta, Rajgopal Kannan, Viktor PrasannaMulti-Agent Coordination LLM-Guided RL
University of Southern California, Los Angeles, California, United States · DEVCOM ARL Army Research Oﬀice, Los Angeles, California, United States
GRAIL: Autonomous Concept Grounding for Neuro-Symbolic Reinforcement Learning
Apr 18, 2026Hikaru Shindo, Henri Rößler, Quentin Delfosse +1Reinforcement Learning LLM-Guided RL
Technical University of Darmstadt · Google Intrinsic · German Research Center for Artificial Intelligence (DFKI) +1
Chain of Uncertain Rewards with Large Language Models for Reinforcement Learning
Apr 15, 2026Shentong MoReward Design for RL LLM-Guided RL
Carnegie Mellon University
KnowRL: Boosting LLM Reasoning via Reinforcement Learning with Minimal-Sufficient Knowledge Guidance
Apr 14, 2026Linhao Yu, Tianmeng Yang, Siyu Ding +8RL for Language Model Reasoning LLM Reasoning
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · Baidu Inc. · Institute of Information Engineering, Chinese Academy of Sciences.
When Valid Signals Fail: Regime Boundaries Between LLM Features and RL Trading Policies
Apr 13, 2026Zhengzhe YangLLM-Guided RL Reinforcement Learning
Independent Researcher
C $^2$ T: Captioning-Structure and LLM-Aligned Common-Sense Reward Learning for Traffic--Vehicle Coordination
Apr 10, 2026Yuyang Chen, Kaiyan Zhao, Yiming Wang +3Traffic Signal Control Multi-Agent Coordination
The State Key Laboratory of Internet of Things for Smart City, University of Macau · Wuhan University · Computer and Information Science, University of Macau +1
ASPECT:Analogical Semantic Policy Execution via Language Conditioned Transfer
Apr 9, 2026Ajsal Shereef Palattuparambil, Thommen George Karimpanal, Santu RanaZero-Shot RL Zero-Shot Policy Transfer
Applied Artificial Intelligence Initiative, Deakin University, Waurn Ponds, Geelong · School of IT, Deakin University, Waurn Ponds, Geelong
3DrawAgent: Teaching LLM to Draw in 3D with Early Contrastive Experience
Apr 9, 2026Hongcan Xiao, Xinyue Xiao, Yilin Wang +23D Spatial Reasoning RL for Generative Models
Beijing University of Posts and Telecommunications · Jiangnan University · HaoHan Data
CuraLight: Debate-Guided Data Curation for LLM-Centered Traffic Signal Control
Apr 7, 2026Qing Guo, Xinhang Li, Junyu Chen +4Traffic Signal Control Language Model-Based Control
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, Beijing, China · Beijing Big Data Center, Beijing, China
When Adaptive Rewards Hurt: Causal Probing and the Switching-Stability Dilemma in LLM-Guided LEO Satellite Scheduling
Apr 4, 2026Yuanhang LiReward Design for RL LLM-Guided RL
When to ASK: Uncertainty-Gated Language Assistance for Reinforcement Learning
Apr 2, 2026Juarez Monteiro, Nathan Gavenski, Gianlucca Zuin +1Uncertainty Estimation in RL Reinforcement Learning
Kunumi Institute Belo Horizonte, MG, Brazil · King’s College London London, England, UK
Learning to Hint for Reinforcement Learning
Apr 1, 2026Yu Xia, Canwen Xu, Zhewei Yao +2Reinforcement Learning RL for Language Model Reasoning
University of California, San Diego · Snowflake AI Research
Learning to Play Blackjack: A Curriculum Learning Perspective
Mar 31, 2026Amirreza Alasti, Efe Erdal, Yücel Celik +1Curriculum RL LLM-Guided RL
Leibniz University Hannover, Hannover, Germany · Leibniz AI Academy, Hannover, Germany
Generalizable Dense Reward for Long-Horizon Robotic Tasks
Mar 31, 2026Silong Yong, Stephen Sheng, Carl Qi +6Reinforcement Learning Long-Horizon Robotic Manipulation
Carnegie Mellon University · Amazon Robotics · UT Austin
Evolutionary Discovery of Reinforcement Learning Algorithms via Large Language Models
Mar 30, 2026Alkis Sygkounas, Amy Loutfi, Andreas PerssonAutomated Algorithm Discovery Evolutionary RL
Machine Perception and Interaction Lab, Örebro University, Sweden
COvolve: Adversarial Co-Evolution of Large-Language-Model-Generated Policies and Environments via Two-Player Zero-Sum Game
Mar 30, 2026Alkis Sygkounas, Rishi Hazra, Andreas Persson +2Lifelong Learning Agents Self-Evolving Agents
Machine Perception and Interaction Lab, Örebro University, Sweden
Beyond Where to Look: Trajectory-Guided Reinforcement Learning for Multimodal RLVR
Mar 27, 2026Jinda Lu, Junkang Wu, Jinghan Li +6RL for VLMs Multimodal Reasoning
University of Science and Technology of China · Pecking University · Institute of Software, Chinese Academy of Sciences
LATS: Large Language Model Assisted Teacher-Student Framework for Multi-Agent Reinforcement Learning in Traffic Signal Control
Mar 25, 2026Yifeng Zhang, Peizhuo Li, Tingguang Zhou +2Traffic Signal Control LLM-Guided RL
Department of Mechanical Engineering, National University of Singapore
Large Language Model Guided Incentive Aware Reward Design for Cooperative Multi-Agent Reinforcement Learning
Mar 25, 2026Dogan Urgun, Gokhan GungorReward Design for RL Cooperative MARL
Department of Electrical and Electronics Engineering, Karabuk University, 78050 Karabuk, Türkiye · Department of Mechatronics Engineering, Karabuk University, 78050 Karabuk, Türkiye
Bridging Network Fragmentation: A Semantic-Augmented DRL Framework for UAV-aided VANETs
Mar 19, 2026Gaoxiang Cao, Wenke Yuan, Huasen He +4Intelligent Transportation Systems Wireless Communications
Department of Automation, University of Science & Technology of China
DriveVLM-RL: Neuroscience-Inspired Reinforcement Learning with Vision-Language Models for Safe and Deployable Autonomous Driving
Mar 18, 2026Zilin Huang, Zihao Sheng, Zhengyang Wan +4VLMs for Autonomous Driving LLM-Guided RL
Department of Civil and Environmental Engineering, University of Wisconsin-Madison, Madison, WI, 53706, USA · Lyles School of Civil and Construction Engineering, Purdue University, West Lafayette, IN 47907, USA · Department of Civil Engineering, McGill University, Montreal, QC, H3A 0C3, Canada
Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control
Mar 18, 2026Hao Ma, Zhiqiang Pu, Xiaolin Ai +1LLM-Guided RL RL Exploration
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · JD.com, Beijing, China

May 5, 2026J/K move · Enter open · S save

Discovering Reinforcement Learning Interfaces with Large Language Models

Akshat Singh Jaswal, Ashish Baghel, Paras Chopra

Lossfunk

Reinforcement Learning LLM-Guided RL Reward Design for RL

Open paper AskPDF arXiv

Reinforcement learning systems rely on environment interfaces that specify observations and reward functions, yet constructing these interfaces for new tasks often requires substantial manual effort. While recent work has automated reward design using large language models (LLMs), these approaches assume fixed observations and do not address the broader challenge of synthesizing complete task interfaces. We study RL task interface discovery from raw simulator state, where both observation mappings and reward functions must be generated. We propose LIMEN (Code available at https://github.com/Lossfunk/LIMEN), a LLM guided evolutionary framework that produces candidate interfaces as executable programs and iteratively refines them using policy training feedback. Across novel discrete gridworld tasks and continuous control domains spanning locomotion and manipulation, joint evolution of observations and rewards discovers effective interfaces given only a trajectory-level success metric, while optimizing either component alone fails on at least one domain. These results demonstrate that automatic construction of RL interfaces from raw state can substantially reduce manual engineering and that observation and reward components often benefit from co-design, as single-component optimization fails catastrophically on at least one domain in our evaluation suite.