Sample-Efficient RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 168

All topics

NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning
Jun 19, 2026Xinwei Liu, Junyuan Liang, Zicong Hong +2Reinforcement Learning Q-Learning
Sun Yat-sen University, China · EPFL, Switzerland · Purdue University, USA
Direct Advantage Estimation for Scalable and Sample-efficient Deep Reinforcement Learning
Jun 18, 2026Hsiao-Ru Pan, Bernhard SchölkopfPartially Observable RL Sample-Efficient RL
Max Planck Institute for Intelligent Systems, Tübingen · ELLIS Institute Tübingen · ETH Zürich
DF-ExpEnse: Diffusion Filtered Exploration for Sample Efficient Finetuning
Jun 17, 2026Calvin Luo, Chen Sun, Shuran SongRobot Policy Adaptation Robotic RL
1Stanford University · 2Brown University.
UBP2: Uncertainty-Balanced Preference Planning for Efficient Preference-based Reinforcement Learning
Jun 17, 2026Mohamed Nabail, Leo Kaixuan Cheng, Jingmin Wang +1Pairwise Preference Evaluation RL Exploration
Learning, Embodied Autonomy, and Forecasting (LEAF) Lab, University of Toronto
RODS: Reward-Driven Online Data Synthesis for Multi-Turn Tool-Use Agents
Jun 17, 2026Ruishan Fang, Siyuan Lu, Chenyi Zhuang +1Synthetic Data Generation Sample-Efficient RL
Inclusion AI, Ant Group · Zhejiang University · Westlake University +1
Performance-Driven Environment Abstraction with Multi-Timescale Learning
Jun 16, 2026Yue Guan, Dipankar Maity, Panagiotis TsiotrasMarkov Decision Processes Reinforcement Learning
School of Aerospace Engineering, Georgia Institute of Technology, Atlanta, GA, USA · Department of Electrical and Computer Engineering, University of North Carolina at Charlotte, Charlotte, NC, USA
Learning Policy from a Single Trajectory in Average-Reward Markov Decision Process
Jun 15, 2026Jongmin Lee, Ernest K. Ryu, Vaneet AggarwalMarkov Decision Processes Reinforcement Learning
Seoul National University · UCLA · Purdue University
Infant Spontaneous Movement Noise Improves Exploration in Deep RL
Jun 15, 2026Francisco M. López, Markus R. Ernst, Francisco Cruz +2Reinforcement Learning RL Exploration
Frankfurt Institute for Advanced Studies, Germany, and with the School of Computer Science and Engineering, University of New South Wales, Australia · School of Computer Science and Engineering, University of New South Wales, Australia · School of Computer Science and Engineering, University of New South Wales, Australia, and with Escuela de Ingenier´ıa, Universidad Central de Chile, Chile +2
Lyapunov-Based Sample Complexity Analysis for Weakly-Coupled MDPs
Jun 12, 2026Tianhao Wu, Matthew Zurek, Weina Wang +1Multi-Armed Bandits Robust Markov Decision Processes
Department of Industrial and Systems Engineering, University of Wisconsin-Madison · Department of Computer Sciences, University of Wisconsin-Madison · Computer Science Department, Carnegie Mellon University
TRACE: A Unified Rollout Budget Allocation Framework for Efficient Agentic Reinforcement Learning
Jun 9, 2026Heming Zou, Qi Wang, Yun Qu +9Agentic RL Reinforcement Learning with Verifiable Rewards
1Tsinghua University · 2LLM Department, Tencent
Representation-Aware Advantage Estimation: Your Reward Model Provides More Than A Scalar Output
Jun 9, 2026Guozheng Li, Xiyan Fu, Yiwen GuoRL for Language Models Sample-Efficient RL
Southeast University · Nanyang Technological University · Independent Researcher
3SPO: State-Score-Supervised Policy Optimization for LLM Agents
Jun 8, 2026Yu Han, Kailing Li, Yang Jiao +4Credit Assignment in RL Step-Level Credit Assignment in RL
School of Computer Science and Technology, East China Normal University · Fudan University · KAUST +1
sGPO: Trading Inference FLOPs for Training Efficiency in RLVR
Jun 7, 2026Shivchander Sudalairaj, Kai Xu, Akash Srivastava +1Group Relative Policy Optimization Adaptive Sampling
AI Innovation, Red Hat · Core AI, IBM
Cross-Epoch Adaptive Rollout Optimization for RL Post-Training
Jun 4, 2026Yiming Zong, Yige Wang, Jiashuo JiangSequential Decision Making RL for Language Model Reasoning
Department of Industrial Engineering & Decision Analytics, Hong Kong University of Science and Technology
Rollout-Level Advantage-Prioritized Experience Replay for GRPO
Jun 3, 2026Gyeongtae Yoo, Sanghyeok Park, Soohyuk Jang +2RL for Language Model Reasoning Group Relative Policy Optimization
Department of Electrical and Computer Engineering, Seoul National University · Interdisciplinary Program in AI, Seoul National University · AIIS, ASRI, INMC, and ISRC, Seoul National University
RLVR without Ineffective Samples: Group Prioritized Off-Policy Optimization for LLM Reasoning
May 31, 2026Yixiu Mao, Yun Qu, Qi Wang +2Reinforcement Learning RL for Language Model Reasoning
Department of Automation, Tsinghua University
Lagrangian Perturbation Diffusion Steering: Latent Reinforcement Learning for Generative Policies
May 31, 2026Hikmet Simsir, Ozgur S. OguzRobot Policy Adaptation Reinforcement Learning
Department of Computer Engineering, Bilkent University, Ankara, Türkiye.
Reinforcement Learning with Pairwise Preferences in Long-Term Decision Problems
May 29, 2026Jonathan Colaço Carr, Prakash Panangaden, Doina Precup +1Pairwise Preference Learning Markov Decision Processes
Stanford University · School of Computer Science, McGill University, Montreal, Quebec, Canada · Mila - Quebec AI Institute, Montreal, Quebec, Canada +1
Sample-Efficient Post-Training for LEGO Spatial-Physics Reasoning
May 29, 2026Yuhuan Yuan, Zhouliang Yu, Minghao Liu +2Physical Reasoning Visual Spatial Reasoning
HKUST(GZ) · CUHK · ZODA
Sample-Efficient Diffusion-based Reinforcement Learning with Critic Guidance
May 28, 2026Shutong Ding, Zejia Zhong, Zhongyi Wang +4Diffusion Guidance Robotic RL
1ShanghaiTech University.
Optimal Data Acquisition for Reinforcement Learning: A Large Deviations Perspective
May 27, 2026Mingjie Hu, Jian-Qiang Hu, Enlu ZhouReinforcement Learning Sample-Efficient RL
School of Management, Fudan University · H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology
IRDS: Interpretable RLVR Data Selection via Verifier-Coupled Sparse Autoencoder Coverage
May 27, 2026Yuhan Li, Mingxu Zhang, Dazhong Shen +1Reinforcement Learning RL for Language Model Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Nanjing University of Aeronautics and Astronautics · The 63rd Research Institute, National University of Defense Technology, Nanjing
BASIS: Batchwise Advantage Estimation from Single-Rollout Information Sharing for LLM Reasoning
May 26, 2026Shijin Gong, Erhan Xu, Kai Ye +3RL for Language Model Reasoning Critic-Free RL
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · Department of Engineering Science, University of Oxford
Ratio-Variance Regularized Policy Optimization
May 26, 2026Yu Luo, Shuo Han, Yihan Hu +5RL for Language Model Reasoning Policy Optimization
Department of Foundation Model, 2012 Labs, Huawei Shanghai · Shanghai Research Institute for Intelligent Autonomous Systems, Tongji University · Department of Computer Science and Technology, Tsinghua University +1
Sample Complexity of Policy Gradient for Log-Growth Control
May 26, 2026Qiuhua Pan, Yukai Shen, Liwei Zhang +2Stochastic Linear Dynamical Systems Stochastic Optimal Control
State Key Laboratory of Submarine Geoscience, School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China · Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai 200240, China · Shanghai Key Laboratory of Perception and Control in Industrial Network Systems, Shanghai 200240, China +1
EXPO-FT: Sample-Efficient Reinforcement Learning Finetuning for Vision-Language-Action Models
May 25, 2026Perry Dong, Kuo-Han Hung, Tian Gao +2Robot Skill Learning Robotic RL
Stanford University
ParkingWorld: End-to-End Autonomous Parking Reinforcement Learning from Corrective Experience in 3DGS Simulation
May 24, 2026Zhengcheng Yu, Changze Li, Haoran Liu +1End-to-End Autonomous Driving Autonomous Parking
Tsinghua University · Shanghai Jiaotong University
Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotion
May 24, 2026Gianluca Sabatini, Chenhao Li, Marco HutterRL for Legged Locomotion Soft Actor-Critic
ETH Zurich, Switzerland
Refined Analysis of Entropy-Regularized Actor-Critic
May 23, 2026Safwan Labbi, Paul Mangold, Daniil Tiapkin +1Reinforcement Learning Maximum Entropy RL
CMAP, CNRS, École Polytechnique, Institut Polytechnique de Paris, 91120 Palaiseau, France · Université Paris-Saclay, CNRS, LMO, 91405, Orsay, France · Google DeepMind +2
Goal-Conditioned Agents that Learn Everything All at Once
May 22, 2026Michael Matthews, Matthew Jackson, Michael Beukman +5Policy Learning Sample-Efficient RL
University of Oxford · 2McGill University · 3MIT +1

Jun 19, 2026, cs.LGJ/K move · Enter open · S save

NASDAQ: Normalized Observation Space Dynamics-Augmented Q-Learning

Xinwei Liu, Junyuan Liang, Zicong Hong, Jianting Zhang+1

Sun Yat-sen University, China · EPFL, Switzerland · Purdue University, USA

Reinforcement Learning Q-Learning Model-Free RL Sample-Efficient RL

Open paper AskPDF arXiv

Augmenting model-free reinforcement learning (RL) with representations learned through observation dynamics prediction (observation-predictive RL) can improve sample efficiency and performance, with minor modifications and limited additional computation. However, this approach still struggles in challenging tasks with low-dimensional observations. In this paper, we identify a key factor behind this problem: unbalanced reconstruction losses across observation dimensions, where dimensions with larger value ranges dominate the loss. This encourages the agent to neglect dimensions with relatively small ranges, leading to degraded performance. To address this issue, we propose a novel normalization method tailored to online RL, which normalizes low-dimensional observations and balances the resulting losses and gradients. Beyond balancing reconstruction losses, observation normalization enables dynamics prediction to be performed in a normalized observation space, thereby providing a unified treatment of low- and high-dimensional inputs (e.g., physical states and images). Building on this idea, we further introduce Normalized Observation Space Dynamics-Augmented Q-learning (NASDAQ), a framework for observation-predictive RL applicable across diverse domains. NASDAQ learns state-action representations by coupling value learning with two auxiliary tasks: short-term value prediction and next normalized observation prediction. Extensive experiments demonstrate that NASDAQ achieves competitive or superior performance compared with state-of-the-art model-based and self-predictive RL methods, while requiring significantly less training wall-time.

Momentum

Related topics

Latest papers 168