Sample-Efficient RL

RL: Reinforcement Learning

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 140% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 168

All topics

Counter-Dyna: Data-Efficient RL-Based HVAC Control using Counterfactual Building Models
May 6, 2026Jan Marco Ruiz de Vargas, Fabian Raisch, Zoltan Nagy +2Neural Surrogate Modeling Building Energy Management
Technical University of Munich Munich, Germany · Technical University of Applied Sciences Rosenheim Rosenheim, Germany · Eindhoven University of Technology Eindhoven, Netherlands +1
Data-dependent Exploration for Online Reinforcement Learning from Human Feedback
May 6, 2026Zhen-Yu Zhang, Yuting Tang, Jiandong Zhang +2RL Exploration Sample-Efficient RL
Center for Advanced Intelligence Project, RIKEN · Graduate School of Frontier Sciences, The University of Tokyo · Northeastern University +1
Optimal Posterior Sampling for Policy Identification in Tabular Markov Decision Processes
May 5, 2026Cyrille Kone, Kevin JamiesonThompson Sampling Markov Decision Processes
Univ. Lille, CNRS, Inria, Centrale Lille, UMR 9189-CRIStAL, F-59000 Lille, France · University of Washington, Seattle, USA
Taming the Curses of Multiagency in Robust Markov Games with Large State Space through Linear Function Approximation
May 4, 2026Jingchu Gai, Laixi ShiMulti-Agent System Optimization Markov Models
Machine Learning Department, Carnegie Mellon University. · Department of Electrical and Computer Engineering, Johns Hopkins University, MD 21286, USA.
Do We Really Need Immediate Resets? Rethinking Collision Handling for Efficient Robot Navigation
May 4, 2026Shanze Wang, Xinming Zhang, Siwei Cheng +4RL for Robotics Robot Navigation
College of Information Science and Technology, Eastern Institute of Technology, Ningbo, China · Department of Aeronautical and Aviation Engineering, The Hong Kong Polytechnic University, Hung Hom, Kowloon, Hong Kong · School of Computer Science and Technology, University of Science and Technology of China, Hefei, China +2
Dynamics Distillation for Efficient and Transferable Control Learning
May 2, 2026Xunjiang Gu, Kashyap Chitta, Mahsa Golchoubian +2Vehicle Dynamics Transfer Learning
University of Toronto · Vector Institute · NVIDIA +1
Breaking the Computational Barrier: Provably Efficient Actor-Critic for Low-Rank MDPs
May 2, 2026Ruiquan Huang, Donghao Li, Yingbin Liang +1Markov Decision Processes Reinforcement Learning
University of Kentucky. · University of Virginia. · Ohio State University..
E $^2$ DT: Efficient and Effective Decision Transformer with Experience-Aware Sampling for Robotic Manipulation
Apr 30, 2026Kaiyan Zhao, Borong Zhang, Yiming Wang +4Long-Horizon Robotic Manipulation Transformer
School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Lab of Internet of Things for Smart City and Department of Computer Information Science, University of Macau, Macao 999078, China
Kernelized Advantage Estimation: From Nonparametric Statistics to LLM Reasoning
Apr 30, 2026Shijin Gong, Kai Ye, Jin Zhu +3Value Function Estimation RL for Language Model Reasoning
School of Management, University of Science and Technology of China · Department of Statistics, London School of Economics and Political Science · School of Mathematics, University of Birmingham +2
Rule-based High-Level Coaching for Goal-Conditioned Reinforcement Learning in Search-and-Rescue UAV Missions Under Limited-Simulation Training
Apr 29, 2026Mahya Ramezani, Holger VoosReinforcement Learning UAV Navigation
Automation and Robotics Research Group, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg.
Sample-efficient Neuro-symbolic Proximal Policy Optimization
Apr 28, 2026Simone Murari, Celeste Veronese, Daniele MeliReinforcement Learning Proximal Policy Optimization
Dept. of Computer Science, University of Verona, Italy
ReCast: Recasting Learning Signals for Reinforcement Learning in Generative Recommendation
Apr 24, 2026Peiyan Zhang, Hanmo Liu, Chengxuan Tong +3RL for Generative Models Sparse-Reward RL
1Huawei Technologies Co., Ltd.
Replay-buffer engineering for noise-aware quantum circuit optimization
Apr 23, 2026Akash Kundu, Sebastian FeldReinforcement Learning Experience Replay
Delft University of Technology, Delft, The Netherlands · Quantum Computing Division, QuTech, The Netherlands
Distributional Value Estimation Without Target Networks for Robust Quality-Diversity
Apr 22, 2026Behrad Koohy, Jamie BayneDistributional RL Reinforcement Learning
Luffy.AI Culham, Oxfordshire, United Kingdom
Planning in entropy-regularized Markov decision processes and games
Apr 21, 2026Jean-Bastien Grill, Omar Darwiche Domingues, Pierre Ménard +2Value Function Estimation Markov Models
DeepMind Paris · SequeL team, Inria Lille
Learning from Less: Measuring the Effectiveness of RLVR in Low Data and Compute Regimes
Apr 20, 2026Justin Bauer, Thomas Walshe, Derek Pham +4RL for Language Model Reasoning Small Language Models
Snorkel AI · University of Wisconsin-Madison · University of Oxford
Scale-free adaptive planning for deterministic dynamics & discounted rewards
Apr 20, 2026Peter L. Bartlett, Victor Gabillon, Jennifer Healey +1Markov Decision Processes Model-Based Planning
University of California, Berkeley, USA · Noah’s Ark Lab, Huawei Technologies, London, UK · Adobe Research, San Jose, USA +1
Freshness-Aware Prioritized Experience Replay for LLM/VLM Reinforcement Learning
Apr 18, 2026Weiyu Ma, Yongcheng Zeng, Yan Song +4Reinforcement Learning RL Post-Training
King Abdullah University of Science and Technology (KAUST) · Chinese Academy of Sciences, Institute of Automation (CASIA) · AI Centre, Department of Computer Science, University College London +1
Sample Complexity Bounds for Stochastic Shortest Path with a Generative Model
Apr 17, 2026Jean Tarbouriech, Matteo Pirotta, Michal Valko +1Markov Decision Processes Policy Learning
Facebook AI Research Paris & Inria Lille · Facebook AI Research Paris · DeepMind Paris
Blazing the trails before beating the path: Sample-efficient Monte-Carlo planning
Apr 16, 2026Jean-Bastien Grill, Michal Valko, Rémi MunosMarkov Decision Processes Markov Models
SequeL team, INRIA Lille - Nord Europe, France · Google DeepMind, UK∗
Provably Efficient Offline-to-Online Value Adaptation with General Function Approximation
Apr 15, 2026Shangzhe Li, Weitong ZhangOffline-To-Online RL Sample-Efficient RL
UNC Chapel Hill
Value Mirror Descent for Reinforcement Learning
Apr 7, 2026Zhichao Jia, Guanghui LanMarkov Decision Processes Reinforcement Learning
H. Milton Stewart School of Industrial and Systems Engineering, Georgia Institute of Technology, Atlanta, GA, 30332.
TRACE: Capability-Targeted Agentic Training
Apr 7, 2026Hangoo Kang, Tarun Suresh, Jon Saad-Falcon +1LLM Agent Self-Improvement Agent Skill Learning
Stanford University
Unsupervised Behavioral Compression: Learning Low-Dimensional Policy Manifolds through State-Occupancy Matching
Mar 27, 2026Andrea Fraschini, Davide Tenedini, Riccardo Zamboni +2RL Control Distribution Matching
Politecnico di Milano · Technion
Train at Moving Edge: Online-Verified Prompt Selection for Efficient RL Training of Large Reasoning Model
Mar 26, 2026Jiahao Wu, Ning Lu, Shengcai Liu +6RL for Language Models Sample-Efficient RL
Southern University of Science and Technology · The Hong Kong Polytechnic University · The Hong Kong University of Science and Technology +3
End-to-End Efficient RL for Linear Bellman Complete MDPs with Deterministic Transitions
Mar 24, 2026Zakaria Mhammedi, Alexander Rakhlin, Nneka OkoloReinforcement Learning Linear Markov Decision Processes
Google Research, NYC · MIT
Complementary RL: Towards Efficient Experience-Driven Agent Learning
Mar 18, 2026Dilxat Muhtar, Jiashun Liu, Wei Gao +8Reinforcement Learning Sample-Efficient RL
Alibaba Group · HKUST
From Prior to Pro: Efficient Skill Mastery via Distribution Contractive RL Finetuning
Mar 10, 2026Zhanyi Sun, Shuran SongRL for Robotics Robot Policy Learning
1Stanford University.
GIPO: Gaussian Importance Sampling Policy Optimization
Mar 4, 2026Chengxuan Lu, Zhenquan Zhang, Shukuan Wang +3Reinforcement Learning RL Post-Training
1IROOTECH TECHNOLOGY · 2Wolf 1069 b Lab, Sany Group
VLM-Guided Experience Replay
Feb 2, 2026Elad Sharony, Tom Jurgenson, Orr Krupnik +2Reinforcement Learning Sample-Efficient RL
Technion · ForSight Robotics · Nvidia Research

May 6, 2026, cs.LGJ/K move · Enter open · S save

Counter-Dyna: Data-Efficient RL-Based HVAC Control using Counterfactual Building Models

Jan Marco Ruiz de Vargas, Fabian Raisch, Zoltan Nagy, Pierre Pinson+1

Technical University of Munich Munich, Germany · Technical University of Applied Sciences Rosenheim Rosenheim, Germany · Eindhoven University of Technology Eindhoven, Netherlands+1

Neural Surrogate Modeling Building Energy Management Feedback Control Sample-Efficient RL Model-Based RL Heating, Ventilation, and Air Conditioning Control

Open paper AskPDF arXiv

Model-based reinforcement learning (MBRL) offers a promising approach for data-efficient energy management in buildings, combining the strengths of predictive modeling and reinforcement learning. While previous MBRL methods applied to HVAC control have reduced training data requirements, they still require several months of interaction with the building to learn a satisfactory control policy. A key reason is that existing surrogate models attempt to predict the entire state-space, including weather and electricity prices that are unaffected by control actions, or completely ignore these variables. Addressing these issues, we propose Counter-Dyna, a method that enhances the data-efficiency of Dyna, an MBRL method. We create data-efficient counterfactual surrogate models (CSM) by leveraging invariances in the state-space. Using a CSM in Dyna speeds up RL training measured in environment interaction data compared to previous results. In comparison with previous state-of-the-art that used 6-12 months of environment interactions, our method needs only 5 weeks. We evaluate our method in a large simulation study using the literature standard BOPTEST framework and proximal policy algorithm (PPO) as the RL algorithm. Our results show cost-saving potentials of 5.3% to 17.0% in a hypothetical deployment scenario. Our work is a significant step towards making real-world deployment of RL algorithms in HVAC control practically viable.

Momentum

Related topics

Latest papers 168