Soft Actor-Critic

Also known as SAC

Soft Actor-Critic (SAC) learns continuous actions by combining reward with an entropy bonus, encouraging exploration, while replay buffers and twin critics support sample-efficient off-policy training. Research builds on this core with distributional critics, ensembles, hierarchical policies, safety constraints, and guidance to improve value estimates, exploration, and long-horizon planning. SAC is studied in settings including robotic locomotion and manipulation, building control, and autonomous driving, with sim-to-real transfer an important goal. A central tension is improving data efficiency and scaling to complex tasks without unstable or overestimated critics, while choosing enough exploration to learn useful behavior.

Latest papers

28 papers4 papers in the last four weeks, up 33% on the four weeks before. 0.0% of all new papers.

Large language model and reinforcement learning based autonomous construction and progress updates
Oct 16, 2026Yifei Xiao, Huasheng Lin, T. Y. YangRobotic Construction Reinforcement Learning
Department of Civil Engineering, The University of British Columbia, Vancouver, Canada
A Geometric Approach to Soft Actor-Critic with Zonotopes for Locomotion Learning
Oct 8, 2026Panagiotis Roditis, Panagiotis P. Filntisis, Petros MaragosReinforcement Learning Soft Actor-Critic
Robotics Institute, Athena Research Center, Marousi, Greece · HERON - Hellenic Robotics Center of Excellence, Athens, Greece · School of Electrical & Computer Engineering, NTUA, Greece
Arithmetic Actor Heads and Training Stabilization for Out-of-Distribution Reinforcement Learning
Oct 4, 2026Yifan Zhang, Liang Zheng, Qifan ZhangOOD Generalization Soft Actor-Critic
Central South University
An analysis of Mirror-Descent Soft Actor-Critic
Sep 28, 2026Denis Zorba, Michal ValkoReinforcement Learning Online Mirror Descent
School of Mathematics, University of Edinburgh, United Kingdom · INRIA, Paris, France
Rank Collapse Is Recoverable, Growing $|Q|$ Is Not: Out-of-Sample Early Warning for Value Divergence in High-UTD Soft Actor-Critic
Sep 26, 2026Tianqi Bu, YuXuan Peng, Junteng Tu +1Off-Policy RL Soft Actor-Critic
Unthrottling the Tanh Jacobian in SAC: A Negative Result on Bang-Bang Control and MetaDrive
Sep 8, 2026Faiq ShamassRL Control Soft Actor-Critic
Independent Researcher
Behavioral Cloning Outperforms Entropy-Regularized RL: Critic-Driven Failure of Actor-Critic Methods on Adaptive Tumor Treatment
Sep 6, 2026Aleksandar Dimitrov, Giacomo SpiglerActor-Critic Methods Soft Actor-Critic
Department of Cognitive Science & Artificial Intelligence, Tilburg University, Tilburg, The Netherlands
WarpSAC: Towards the Pinnacle of Scalable Off-policy RL by Rethinking Exploration and Exploitation
Aug 25, 2026Zihao Wu, Hongyao Tang, Yi Ma +9Reinforcement Learning Off-Policy RL
Tianjin University · Shanxi University · Imperial College London
Spiking Neural Networks for Continuous Control: Neuromorphic Reinforcement Learning in Conventional Computing
Aug 24, 2026Jessica Hunter, Md Maruf Hossain Shuvo, Krishna RoyReinforcement Learning Spiking Neural Networks
Department of Computer Science New Mexico Institute of Mining and Technology Socorro, NM 87801, USA · Department of Electrical and Computer Engineering The University of Texas at El Paso El Paso, TX 79968, USA · Department of Electrical Engineering New Mexico Institute of Mining and Technology Socorro, NM 87801, USA
Learning to Modulate, Not to Cycle: Soft Actor---Critic Recovers Inverter-Style Heat-Pump Control
Aug 10, 2026Faizan Ahmed, Aniket Dixit, James BruseyRL Control Feedback Control
Centre for Computational Science and Mathematical Modelling, Coventry University, United Kingdom
LooperMuscle: Fast and Stable Learning of Humanoid Whole-Body Tracking via Structured Mixture-of-Experts
Aug 1, 2026Boyi Liu, Qijin Li, Tianqi Yu +2Robotic RL Humanoid Motion Tracking
1DeepMirror Inc., Guangzhou, China. · 3Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), Abu Dhabi, UAE.
Hierarchical Soft Actor-Critic for Sparse-Reward Long-Horizon Reinforcement Learning
Jul 26, 2026Zahra Abdalla Elashaal, Afef Hfaiedh, Nahla Khraief +2Reinforcement Learning Hierarchical RL
Lab. of Robotics, Informatics and Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia · Faculty of Information Technology, University of Tripoli, Tripoli, Libya · Lab. of Robotics, Informaticsand Complex Systems, National Engineering School of Tunis, University of Tunis El Manar, Tunis, Tunisia +3
Comparative Study of Multi-Agent Actor-Critic Algorithms in Parameterized Action Reinforcement Learning
Jul 21, 2026Ubayd Ali Bapoo, Clement N NyirendaReinforcement Learning Soft Actor-Critic
Department of Computer Science University of the Western Cape Cape Town, South Africa
Active Beyond-Diagonal RIS Empowered Heterogeneous Edge Computing: A Distributional Reinforcement Learning Approach
Jul 14, 2026Tianyu Pang, Hongyu LiDistributional RL Reconfigurable Intelligent Surfaces
Internet of Things Thrust, The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
Multi-Head Attention-Based Feature Extractor Integration with Soft Actor-Critic for Porosity Prediction and Process Parameter Optimization in Additive Manufacturing
Jun 18, 2026Kianoush Aqabakee, Leonardo StellaMaterials Property Prediction Soft Actor-Critic
Department of Electrical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran, and Department of Mechanical Engineering, Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran · School of Computer Science, University of Birmingham, Birmingham B15 2TT, United Kingdom
Quantifying the Energy Floor: Direct Measurement and Replay Buffer Bias in SAC-Based HVAC Control on sbsim
Jun 1, 2026Bo Li, Chen ZhangExperience Replay Building Energy Management
Shanghai Jiao Tong University College of Smart Energy Shanghai, China
Adaptive Reinforcement Learning for Robust Open Quantum System Control: A Multi-Task Framework with Temporal Optimization
May 26, 2026Haftu W. Fentaw, Steve Campbell, Simon CatonRL Control Multi-Task RL
School of Computer Science, University College Dublin, Belfield, Dublin 4, Dublin, Ireland. · Centre for Quantum Engineering, Science, and Technology, University College Dublin, Belfield, Dublin 4, Dublin, Ireland. · School of Physics, University College Dublin, Belfield, Dublin 4, Dublin, Ireland.
Bridging the Gap: Enabling Soft Actor Critic for High Performance Legged Locomotion
May 24, 2026Gianluca Sabatini, Chenhao Li, Marco HutterRL for Legged Locomotion Soft Actor-Critic
ETH Zurich, Switzerland
When Does Adaptive Guidance Help? Belief-Aware Privileged Distillation for Autonomous Driving Under Partial Observability
May 24, 2026Mehmet HaklidirPartially Observable RL Autonomous Driving
TUBITAK BILGEM Artificial Intelligence Institute, Turkey
Revisiting Mixture Policies in Entropy-Regularized Actor-Critic
May 9, 2026Jiamin He, Samuel Neumann, Jincheng Mei +2RL Control Policy Optimization
University of Alberta & Amii · Google DeepMind · Canada CIFAR AI Chair
Distributional Reinforcement Learning via the Cramér Distance
Apr 26, 2026Vanya Aziz, Ivo Nowak, E. M. T HendrixDistributional RL Robotic RL
HAW Hamburg · de Málaga
FlashSAC: Fast and Stable Off-Policy Reinforcement Learning for High-Dimensional Robot Control
Apr 6, 2026Donghu Kim, Youngdo Lee, Minho Park +10Off-Policy RL Robotic RL
Holiday Robotics · KAIST · KRAFTON +5
RE-SAC: Disentangling aleatoric and epistemic risks in bus fleet control: A stable and robust ensemble DRL approach
Mar 19, 2026Yifan Zhang, Liang Zheng, Qifan ZhangUncertainty Estimation in RL Reinforcement Learning
Central South University
Efficient Soft Actor-Critic with LLM-Based Action-Level Guidance for Continuous Control
Mar 18, 2026Hao Ma, Zhiqiang Pu, Xiaolin Ai +1LLM-Guided RL RL Exploration
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · JD.com, Beijing, China
Squint: Fast Visual Reinforcement Learning for Sim-to-Real Robotics
Feb 24, 2026Abdulaziz Almuzairee, Henrik I. ChristensenVisual RL Soft Actor-Critic
Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA 92093, USA
ManeuverNet: A Soft Actor-Critic Framework for Precise Maneuvering of Double-Ackermann-Steering Robots with Optimized Reward Functions
Feb 16, 2026Kohio Deflesselle, Mélodie Daniel, Aly Magassouba +2Robotic RL Agricultural Robotics
Univ. Bordeaux, CNRS, Bordeaux INP, LaBRI, UMR 5800, F-33400 Talence, France. · School of Computer Science, University of Nottingham, UK. · Instituto de Investigación en Ingeniería de Aragón (I3A), Universidad de Zaragoza, 50018 Zaragoza, Spain.
Constrained Reinforcement Learning for Safe Heat Pump Control
Sep 29, 2024Baohe Zhang, Lilli Frison, Thomas Brox +1RL Control Constrained RL
Faculty of Computer Science, University of Freiburg, Germany · Systems Control and Optimization Laboratory, Department of Microsystems Engineering, University of Freiburg, Germany
Enhancing Hardware Fault Tolerance in Machines with Reinforcement Learning Policy Gradient Algorithms
Jul 21, 2024Sheila Schoepp, Mehran Taghian, Shotaro Miwa +3RL Control Reinforcement Learning
Department of Computing Science, University of Alberta, 8900 114 St NW, Edmonton, Alberta, T6G 2S4, Canada · Advanced Technology R&D Center, Mitsubishi Electric Corporation, 8-1-1,Tsukaguchi-honmachi, Amagasaki-shi, Hyogo, 661-8661, Japan · Information Technology R&D Center, Mitsubishi Electric Corporation, 5-1-1, Ofuna, Kamakura-shi, Kanagawa, 247-8501, Japan +1

Oct 16, 2026J/K move · Enter open · S save

Large language model and reinforcement learning based autonomous construction and progress updates

Yifei Xiao, Huasheng Lin, T. Y. Yang

Department of Civil Engineering, The University of British Columbia, Vancouver, Canada

Robotic Construction Reinforcement Learning Soft Actor-Critic

Open paper AskPDF arXiv

Due to the rapid growth of the global population, traditional construction methods are gradually unable to meet the current housing demand. The advancement of machine learning algorithms has encouraged researchers to explore their applications in the construction industry, leading to more intelligent and efficient building processes. In particular, the rise of large language models in recent years has provided more possibilities for the transformation from traditional construction to smart construction. In this study, a large language model and reinforcement learning based autonomous construction method is proposed. An enhanced soft actor-critic reinforcement learning algorithm is developed to plan actions for a mobile crane according to construction tasks. Robot kinematics and control algorithms are used to ensure that the actions generated by the reinforcement learning algorithm can be executed by the crane. In addition, a customized embedding model is developed to extract information from the construction environment according to user interests. Then, a large language model, ChatGPT 5, is used to update construction progress, provide safety information to control system, and generate construction progress reports. The feasibility of proposed method is verified through several full-scall retaining wall construction experiments.