Critic-Free Reinforcement Learning

Ask about this topic

Momentum

7 papers in the last four weeks, up 133% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 34

All topics
CardsList

  1. Trust the Critic More

    Sep 30, 2026Kaiyue Wen, Luke Bailey, Arvind Mahankali +1Critic-Free Reinforcement LearningLarge Language Model Reinforcement Learning

    Stanford University

  2. Unlocking the Critic: Reward-Free Policy Optimization for LLM Post-Training

    Sep 29, 2026Hongyang Li, Xiao Li, Caesar Wu +3Critic-Free Reinforcement LearningReward-Only Policy Optimization Variants

    University of Luxembourg · Seafill Open-Source Community · Université Paris-Saclay

  3. Adjoint Guidance Flow: Amortized Critic Guidance for VLA Policies

    Sep 28, 2026Jeongsol Kim, Youngjun Jun, Kyumin Choi +6Flow-Matching Vision-Language-ActionCritic-Free Reinforcement Learning

    KAIST · SKKU · GIST

  4. PACT: From Credit Assignment to Critic Alignment

    Sep 22, 2026Jiayan Fu, Hang Xu, Yong Zhang +4Critic-Free Reinforcement LearningCredit Assignment

    AllSpark Team

  5. Critical-State RL: Diagnosing Trainable States for Multi-Turn Tool Use

    Sep 21, 2026Zixiang Chen, Wenting Zhao, Zhepeng Cen +9Critic-Free Reinforcement LearningOffline Reinforcement Learning

    Salesforce AI Research

  6. Fork Where the Model Changes Its Mind: Belief-Shift Branching for Tree-Structured Reinforcement Learning

    Sep 12, 2026Bin Lei, Yu Li, Prafulla Kumar Choubey +7Reinforcement Learning With Verifiable RewardOffline Reinforcement Learning

    University of Minnesota · Salesforce AI Research

  7. T1: Terminal Agent Reinforcement Learning for Long-Horizon Tasks

    Sep 12, 2026Junyao Yang, Yucheng Shi, Zhongzhi Li +4Critic-Free Reinforcement LearningLong-Horizon Task Planning

    Tencent Hy Foundation Model · National University of Singapore · University of Georgia +2

  8. CAST: Critique-Aware Supervision for Training Reliable Long-Horizon Tool-Calling Agents

    Aug 31, 2026Amir Saeidi, Zehua Zhang, Rishitosh Singh +6Large Language Model AgentsCritic-Free Reinforcement Learning

    Arizona State University · Cisco Research

  9. Don't Solve, Just Compare: Tiny Advisors for Runtime Intervention in LLM Agents

    Aug 21, 2026Yanze Jiang, Mingxuan Li, Yuhao Wang +2Large Language Model AgentsRuntime

    National University of Singapore

  10. Efficient Real-World Online Reinforcement Learning for Robot Manipulation via Centralized Training and Critic Decomposition

    Aug 10, 2026Changhao Li, Yifang Zhang, Heng Zhang +6Robotic Manipulation PoliciesRobotic Manipulation

    HHCM, Istituto Italiano di Tecnologia, Genoa, Italy · DIBRIS, University of Genova, Genova, Italy · Human-Robot Interfaces and Interaction Lab, Istituto Italiano di Tecnologia, Genova, Italy +1

  11. DiDPO: Diff-in-Diff Policy Optimization for Coding Agent Training

    Aug 7, 2026Xucong Wang, Zhe Zhao, Liheng Yu +3Reinforcement Learning With Verifiable RewardCritic-Free Reinforcement Learning

    University of Science and Technology of China (USTC) · Stanford University · Tongji University +1

  12. Start Classifying: Categorical Critics for LLM Reinforcement Learning

    Aug 3, 2026Zhijian Zhou, Long Li, Xuan Zhang +7Large Language Model Reinforcement LearningCritic-Free Reinforcement Learning

    Fudan University · Shanghai Innovation Institute · Tencent YoutuLab +2

  13. Not All Tokens Deserve Equal Credit: Counterfactual Sensitivity Credit Reallocation for Long-CoT Reasoning

    Jul 30, 2026Qiangqiang He, Zhongheng Wu, ZiJian WangCredit AssignmentReinforcement Learning With Verifiable Reward

    State Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China · Institute of Wireless Communications Technology, Shanghai Jiao Tong University, Shanghai, China

  14. Learning to Detect UI Principle Violations via Reinforcement Learning

    Jul 22, 2026Nishi Mehta, Swathi Alse, Himani Kumawat +2Code QualityInteraction Design

    University of California, Santa Cruz · Carnegie Mellon University

  15. PS-PPO: Prefix-Sampling PPO for Critic-Free RLHF

    Jun 29, 2026Doo Hwan Hwang, Kee-Eung KimReinforcement Learning From Human FeedbackCritic-Free Reinforcement Learning

    Kim Jaechul Graduate School of AI, KAIST, Daejeon, South Korea.

  16. BV-Blend: Uncertainty-Weighted Historical Baselines for Stable Critic-Free RL with Verifiable Rewards

    Jun 27, 2026Yupeng Chang, Yuan Wu, Yi ChangReinforcement Learning With Verifiable RewardCritic-Free Reinforcement Learning

    School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, MOE, China · 3International Center of Future Science, Jilin University

  17. Phi-Actor-Critic: Steering General-Sum Games to Pareto-Efficient Correlated Equilibria

    Jun 9, 2026Wongyu Lee, Francesco Lelli, Omran Ayoub +1Multi-Agent Reinforcement LearningSoft Actor-Critic

    Politecnico di Milano · Tilburg University · University of Applied Sciences and Arts of Southern Switzerland

  18. Teacher-Free Self-Training Amplifies but Does Not Compound: A Pass@KK Crossover on a Free-Verifier Domain

    Jun 5, 2026Igor Lima StrozziSelf-PlayCritic-Free Reinforcement Learning

    Applied Mathematics Department Federal University of Rio de Janeiro

  19. On the Generalization Gap in Self-Evolving Language Model Reasoning

    May 31, 2026Zhenting Qi, Susanna Maria Baby, Stefanie Anna Baby +5Self-EvolutionReasoning Benchmark

    Google Research · Harvard University · Google +1

  20. Drift Q-Learning

    May 29, 2026Anas Houssaini, Mohamad H. Danesh, Amin Abyaneh +3Diffusion-Based Reinforcement Learning MethodsFlow Policies

    McGill University · Mila - Quebec AI Institute

  21. Explicit Critic Guidance for Aligning Diffusion Models

    May 26, 2026Zhengyang Liang, Qihang Zhang, Ceyuan YangDiffusion AlignmentDiffusion-Based Reinforcement Learning Methods

    University of Toronto · Vector Institute · The Chinese University of Hong Kong

  22. Value-Gradient Hypothesis of RL for LLMs

    May 20, 2026Arip Asadulaev, Daniil Ognev, Karim Salta +1Large Language Model Reinforcement LearningCritic-Free Reinforcement Learning

    MBZUAI · Independent

  23. FedCritic: Serverless Federated Critic Learning-based Resource Allocation for Multi-Cell OFDMA in 6G

    May 20, 2026Amin Farajzadeh, Melike Erol-KantarciOrthogonal Frequency-Division MultiplexingOpen Radio Access Networks

    School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada

  24. Iterative Critique-and-Routing Controller for Multi-Agent Systems with Heterogeneous LLMs

    May 9, 2026Wenzhi Fang, Liangqi Yuan, Guangchen Lan +2Multi-Agent Large Language Model SystemsMulti-Agent Coordination

    Purdue University · Yonsei University

  25. The Cancellation Hypothesis in Critic-Free RL: From Outcome Rewards to Token Credits

    May 9, 2026Tianhao Cheng, Zeyu Huang, Zihan Qiu +5Critic-Free Reinforcement LearningCredit Assignment

    1Fudan University · 3The University of Edinburgh · 6Qwen Team, Alibaba Group +4

  26. When Does Critique Improve AI-Assisted Theoretical Physics? SCALAR: Structured Critic--Actor Loop for Agentic Reasoning

    May 7, 2026Vasilis Niarchos, Constantinos Papageorgakis, Alexander G. Stapleton +1Artificial Intelligence ScientistsCritique

    Department of Physics, CCTP and ITCP, University of Crete, 71303, Greece · Centre for Theoretical Physics, Department of Physics and Astronomy, Queen Mary University of London, London E1 4NS, United Kingdom · Theoretical Physics Department, CERN, Geneva, Switzerland +1

  27. Cross-Modal Navigation with Multi-Agent Reinforcement Learning

    May 7, 2026Shuo Liu, Xinzichen Li, Christopher AmatoMultimodal AgentsMulti-Agent Reinforcement Learning

    Khoury College of Computer Sciences Northeastern University Boston, MA, 02120

  28. Soft Deterministic Policy Gradient with Gaussian Smoothing

    May 7, 2026Hyunjun Na, Donghwan LeePolicy GradientValue Functions

    School of Electrical Engineering, KAIST 291 Daehak-ro, Yuseong-gu, Daejeon 34141, Korea

  29. VLA-ATTC: Adaptive Test-Time Compute for VLA Models with Relative Action Critic Model

    May 2, 2026Wenhao Li, Xiu Su, Yichao Cao +5Diffusion-Based Vision-Language-ActionsCritic-Free Reinforcement Learning

    University of Sydney · Central South University · University of Science and Technology of China +2

  30. Exploring Pass-Rate Reward in Reinforcement Learning for Code Generation

    May 1, 2026Xin-Ye Li, Ren-Biao Liu, Yun-Ji Zhang +3Pass-Rate EvaluationCode Generation

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China · School of Artificial Intelligence, Nanjing University, Nanjing 210023, China

  31. Scalable Neighborhood-Based Multi-Agent Actor-Critic

    Apr 20, 2026Tim Goppelsroeder, Rasmus JensenMulti-Agent Reinforcement LearningCritic-Free Reinforcement Learning

    Department of Computer Science, ETH Zürich.

  32. Let Language Constrain Geometry: Vision-Language Models as Semantic and Spatial Critics for 3D Generation

    Nov 18, 2025Weimin Bai, Yubo Li, Weijian Luo +43D Generative Models3D Spatial Reasoning

    Peking University, Beijing · hi-lab, Xiaohongshu Inc. · MMLab, CUHK +1

  33. Transformer-Based Multi-Agent Reinforcement Learning for Networked Systems with Long-Range Interactions

    Nov 17, 2025Vidur Sinha, Muhammed Ustaomeroglu, Guannan QuMulti-Agent Reinforcement LearningCritic-Free Reinforcement Learning

    Department of Electrical and Computer Engineering, Carnegie Mellon University, Pittsburgh, USA

  34. Bringing Value Models Back: Generative Critics for Value Modeling in LLM Reinforcement Learning

    Date pendingZikang Shan, Han Zhong, Liwei Wang +1Large Language Model Reinforcement LearningCritic-Free Reinforcement Learning

    Peking University · Microsoft Research Asia · Shanghai Jiao Tong University