LLM-Guided RL

RL: Reinforcement Learning

This field studies how large language models can help reinforcement-learning agents learn useful behavior, and how reinforcement learning can improve language models’ reasoning and tool use. Common approaches use models to propose or refine rewards, provide selective hints or curricula, generate plans and reusable skills, or supply semantic knowledge that is distilled into policies. Work spans sparse-reward robotics and game environments, multi-agent coordination, autonomous driving, and language-model reasoning tasks. A central challenge is making guidance reliable and transferable: changing or inaccurate rewards can destabilize learning, and gains against model-based judges may reflect reward hacking rather than better real-world behavior.

Latest papers

140 papers19 papers in the last four weeks, up 46% on the four weeks before. 0.1% of all new papers.

LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement
Mar 14, 2026Chih-Ning Chen, Jen-Cheng Hou, Hsin-Min Wang +3LLM-Guided RL Speech Enhancement
Department of Electrical Engineering, National Taiwan University, Taiwan · Research Center for Information Technology Innovation, Academia Sinica, Taiwan · Center for Hearing Research, University of California Irvine, USA
Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
Mar 12, 2026Yixin Liu, Yue Yu, DiJia Su +7LLM-as-a-Judge LLM Alignment
Meta Superintelligence Labs · Yale Univeristy
Novelty Adaptation Through Hybrid Large Language Model (LLM)-Symbolic Planning and LLM-guided Reinforcement Learning
Mar 11, 2026Hong Lu, Pierrick Lorang, Timothy R. Duggan +2Robotics Neuro-Symbolic Reasoning
Department of Computer Science, Tufts University, Medford, MA, USA · Austrian Institute of Technology GmbH, Vienna, Austria
Code-Space Response Oracles: Generating Interpretable Multi-Agent Policies with Large Language Models
Mar 10, 2026Daniel Hennes, Zun Li, John Schultz +1Multi-Agent Reinforcement Learning Interpretability in RL
Google DeepMind
SCALAR: Learning and Composing Skills through LLM Guided Symbolic Planning and Deep RL Grounding
Mar 10, 2026Renos Zabounidis, Yue Wu, Simon Stepputtis +4LLM-Guided RL LLM Agent Skill Learning
School of Computer Science, Carnegie Mellon University · Department of Mechanical Engineering, Virginia Tech
LBM: Hierarchical Large Auto-Bidding Model via Reasoning and Acting
Mar 5, 2026Yewen Li, Zhiyi Lyu, Peng Jiang +4Automated Bidding Real-Time Bidding
Kuaishou Technology Beijing, China · Nanyang Technological University Singapore, Singapore
WebFactory: Automated Compression of Foundational Language Intelligence into Grounded Web Agents
Mar 5, 2026Sicheng Fan, Qingyun Shi, Shengze Xu +5Web Agents LLM Agent Training
Fudan University · IMean AI · The Chinese University of Hong Kong +1
LLMs for High-Frequency Decision-Making: Normalized Action Reward-Guided Consistency Policy Optimization
Mar 3, 2026Yang Zhao, Zihao Li, Zhiyu Jiang +3Policy Learning LLM-Guided RL
School of Artificial Intelligence, OPtics and ElectroNics (iOPEN), Northwestern Polytechnical University, Xi’an 710072, China
LLM-assisted Semantic Option Discovery for Facilitating Adaptive Deep Reinforcement Learning
Mar 2, 2026Chang Yao, Jinghui Qin, Kebing Jin +1Reinforcement Learning LLM-Guided RL
RF-Agent: Automated Reward Function Design via Language Agent Tree Search
Feb 27, 2026Ning Gao, Xiuhui Zhang, Xingyu Jiang +3Reward Design for RL RL Control
Beihang University, 37 Xueyuan Road, Haidian District, Beijing
Memory-Based Advantage Shaping for LLM-Guided Reinforcement Learning
Feb 20, 2026Narjes Nourzad, Carlee Joe-WongMemory-Augmented RL Reinforcement Learning
University of Southern California · Carnegie Mellon University
MIRA: Memory-Integrated Reinforcement Learning Agent with Limited LLM Guidance
Feb 20, 2026Narjes Nourzad, Carlee Joe-WongMemory-Augmented RL Reinforcement Learning
University of Southern California, Electrical and Computer Engineering, Los Angeles, CA 90007, USA · Carnegie Mellon University, Electrical and Computer Engineering, Pittsburgh, PA 15213, USA
Retrospective In-Context Learning for Temporal Credit Assignment with Large Language Models
Feb 19, 2026Wen-Tse Chen, Jiayu Chen, Fahim Tajwar +4Step-Level Credit Assignment in RL Sparse-Reward RL
Carnegie Mellon University · The University of Hong Kong · Stanford University
PhGPO: Pheromone-Guided Policy Optimization for Long-Horizon Tool Planning
Feb 14, 2026Yu Li, Guangfeng Cai, Shengtian Yang +5LLM-Guided RL Tool-Use Planning
School of Computer Science and Engineering, Southeast University, Nanjing, China · Huawei Noah’s Ark Lab, China
Don't Tell the Answer, Truly Guide the Reasoning During RL Rollouts
Oct 10, 2025Xinyi Wang, Jinyi Han, Zishang Jiang +7Reinforcement Learning RL for Language Model Reasoning
School of Data Science, Fudan University · 2Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1
CLARity: Reasoning Consistency Alone Can Teach Reinforced Experts
Oct 10, 2025Jiuheng Lin, Cong Jiang, Zirui Wu +2RL for Language Models RL for Language Model Reasoning
Wangxuan Institute of Computer Technology, Peking University
Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs
Oct 5, 2025Zishang Jiang, Jinyi Han, Tingyun Li +7RL for Language Model Reasoning LLM-Guided RL
School of Data Science, Fudan University · Shanghai Institute of Artificial Intelligence for Education, East China Normal University · College of Computer Science and Artificial Intelligence, Fudan University +1
Self-CriTeach: LLM Self-Teaching and Self-Critiquing for Improving Robotic Planning via Automated Domain Generation
Sep 25, 2025Jinbang Huang, Zhiyuan Li, Yuanzhao Hu +4Large Language Model-Based Robot Planning Synthetic Data Generation for Robotics
Huawei Noah’s Ark Lab · University of Toronto · University of British Columbia +1
Agentic Episodic Control
Jun 2, 2025Xidong Yang, Wenhao Li, Junjie Sheng +4Agentic RL Reinforcement Learning
East China Normal University · Tongji University · Shanghai Jiao Tong University +2
Policy Learning with a Language Bottleneck
May 7, 2024Megha Srivastava, Cedric Colas, Dorsa Sadigh +1Policy Learning LLM-Guided RL
Stanford University · Massachusetts Institute of Technology Inria

Mar 14, 2026J/K move · Enter open · S save

LLM-Guided Reinforcement Learning for Audio-Visual Speech Enhancement

Chih-Ning Chen, Jen-Cheng Hou, Hsin-Min Wang, Shao-Yi Chien+2

Department of Electrical Engineering, National Taiwan University, Taiwan · Research Center for Information Technology Innovation, Academia Sinica, Taiwan · Center for Hearing Research, University of California Irvine, USA

LLM-Guided RL Speech Enhancement

Open paper AskPDF arXiv

In existing Audio-Visual Speech Enhancement (AVSE) methods, objectives such as Scale-Invariant Signal-to-Noise Ratio (SI-SNR) and Mean Squared Error (MSE) are widely used; however, their correlation with perceived speech quality is often suboptimal and provides limited interpretability for optimization. This work proposes a reinforcement learning-based AVSE framework with a Large Language Model (LLM)-based interpretable reward model. An audio LLM generates natural language descriptions of enhanced speech, which are converted by a sentiment analysis model into a 1-5 rating score serving as the PPO reward for fine-tuning a pretrained AVSE model. Compared with scalar metrics, LLM-generated feedback is semantically rich and explicitly describes speech quality improvements. Experiments on the AVSEC-4 dataset show that the proposed method outperforms a supervised baseline and a DNSMOS-based RL baseline in PESQ, STOI, neural quality metrics, and subjective listening tests.