Test-Time RL

RL: Reinforcement Learning

Test-time reinforcement learning studies how models and agents can improve their decisions on encountered problems using feedback available during inference, rather than relying only on a conventional labeled training set. For language models, common approaches use majority-vote pseudo-labels, confidence or external verification, process rewards, and policy-gradient updates; other systems retrieve experience or steer frozen policies with value gradients, search, or execution feedback. These methods are applied to reasoning, code generation, web agents, multimodal tasks, and robot control, drawing feedback from test cases, tools, simulators, or interaction. A central challenge is that self-generated rewards can reinforce wrong consensus, suppress rare correct answers, or destabilize behavior, making reliable verification, exploration, and compute-efficient gains key open concerns.

Latest papers

48 papers10 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification
Jun 2, 2026Jiahui Li, Jianfeng Shan, Wenpei Chen +5Reinforcement Learning LLM Answer Verification
1Sun Yat-Sen University · University of Science and Technology of China · 3National University of Singapore
STEAM: A Training-Free Congestion-Aware Enhancement Framework for Decentralized Multi-Agent Path Finding
May 20, 2026Mingyang Feng, Mengnuo Zhang, Shaoyuan Li +1Multi-Agent System Optimization Multi-Agent Coordination
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai 200240, China
Detecting and Mitigating the Correct-Answer Extinction Window in Test-Time Reinforcement Learning with Majority Voting
May 19, 2026Hongxiang Lin, Zhirui Kuai, Erpeng Xue +1Reinforcement Learning RL for Language Model Reasoning
Meituan
PACEvolve++: Improving Test-time Learning for Evolutionary Search Agents
May 7, 2026Minghao Yan, Bo Peng, Benjamin Coleman +11Test-Time Adaptation Reinforcement Learning
Google · University of Wisconsin–Madison · Google DeepMind
Understanding and Mitigating Spurious Signal Amplification in Test-Time Reinforcement Learning for Math Reasoning
Apr 23, 2026Yongcan Yu, Lingxiao He, Jian Liang +5Reinforcement Learning RL for Language Model Reasoning
1NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · *Work done during an internship at Meituan · School of Artificial Intelligence, University of Chinese Academy of Sciences +2
TEMPO: Scaling Test-time Training for Large Reasoning Models
Apr 21, 2026Qingyang Zhang, Xinke Kong, Haitao Wu +7Reinforcement Learning Inference-Time Optimization
Tianjin University · Tongyi Lab, Alibaba Group · The Chinese University of Hong Kong +1
TTVS: Boosting Self-Exploring Reinforcement Learning via Test-time Variational Synthesis
Apr 9, 2026Sikai Bai, Haoxi Li, Jie Zhang +2Test-Time Adaptation Test-Time RL
The Hong Kong University of Science and Technology
TTA-Vid: Generalized Test-Time Adaptation for Video Reasoning
Apr 1, 2026Soumya Shamarao Jahagirdar, Edson Araujo, Anna Kukleva +7Test-Time Adaptation Video Reasoning
University of Tübingen · Max Planck Institute for Informatics, SIC · MIT +3
TR-ICRL: Test-Time Rethinking for In-Context Reinforcement Learning
Apr 1, 2026Wenxuan Jiang, Yuxin Zuo, Zijian Zhang +8In-Context RL RL for Language Model Reasoning
The Hong Kong Polytechnic University · Meituan-M17 · Institute of Computing Technology, Chinese Academy of Sciences +3
What If Consensus Lies? Selective-Complementary Reinforcement Learning at Test Time
Mar 20, 2026Dong Yan, Jian Liang, Yanbo Wang +3RL for Language Model Reasoning Test-Time RL
School of Artificial Intelligence, University of Chinese Academy of Sciences · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences · Nanjing University
From the Inside Out: Progressive Distribution Refinement for Confidence Calibration
Mar 17, 2026Xizhong Yang, Yinan Xia, Huiming Wang +1Test-Time RL Reinforcement Learning
Southeast University · Kuaishou Technology · SUTD
Meta-TTRL: A Metacognitive Framework for Self-Improving Test-Time Reinforcement Learning in Unified Multimodal Models
Mar 16, 2026Lit Sin Tan, Junzhe Chen, Xiaolong Fu +5Test-Time RL T2I Generation
Tsinghua University
Amplification Effects in Test-Time Reinforcement Learning: Safety and Reasoning Vulnerabilities
Mar 16, 2026Vanshaj Khattar, Md Rafi ur Rashid, Moumita Choudhury +4Test-Time RL LLM Safety
Virginia Tech · Penn State University · University of Massachusetts Amherst +1
Test-time RL alignment exposes task familiarity artifacts in LLM benchmarks
Mar 13, 2026Kun Wang, Reinhard HeckelLLM Evaluation Inference-Time Language Model Alignment
School of Computation, Information and Technology, Technical University of Munich · Munich Center for Machine Learning
Tool Verification for Test-Time Reinforcement Learning
Mar 2, 2026Ruotong Liao, Nikolai Röhrich, Xiaohan Wang +4Test-Time RL RL for Language Model Reasoning
Ludwig-Maximilians University of Munich · MCML · Stanford University
TTCS: Test-Time Curriculum Synthesis for Self-Evolving
Jan 30, 2026Chengyi Yang, Zhishang Xiang, Yunbo Tang +5Test-Time Training Language Model Self-Improvement
Xiamen University · Washington University in St. Louis · Renmin University of China
Just-In-Time Reinforcement Learning: Continual Learning in LLM Agents Without Gradient Updates
Jan 26, 2026Yibo Li, Zijie Lin, Ailin Deng +5Continual Learning for LLM Agents Reinforcement Learning
National University of Singapore, Singapore.
Context Value Informed In-Context Reinforcement Learning
Sep 27, 2025Wenhao Zhang, Shao Zhang, Xihuai Wang +2In-Context RL Test-Time RL
Shanghai Jiao Tong University

Jun 2, 2026J/K move · Enter open · S save

Exploiting Verification-Generation Gap: Test-Time Reinforcement Learning with Confidence-Conditioned Verification

Jiahui Li, Jianfeng Shan, Wenpei Chen, Shunyu Wu+4

1Sun Yat-Sen University · University of Science and Technology of China · 3National University of Singapore

Reinforcement Learning LLM Answer Verification RL for Language Model Reasoning Test-Time RL Reinforcement Learning with Verifiable Rewards

Open paper AskPDF arXiv

Test-time reinforcement learning has emerged as a promising paradigm for enhancing the complex reasoning abilities of large language models in a completely label-free manner. Despite existing studies focusing on Pass@1 performance, optimizing Pass@k remains under-explored yet critical in label-free settings, which measures generation coverage for sustained exploration. Optimizing Pass@k in label-free setting is highly non-trivial, as directly applying the Pass@k advantage designs effective for RLVR yields unsatisfactory performance. Through in-depth empirical analysis, we discover the root causes hindering performance: pseudo-label estimations for low-confidence samples have a high probability of being incorrect, while candidate answers for high-confidence samples suffer from severe diversity collapse. To overcome these hurdles, we propose TTRL-CoCoV (Test-Time Reinforcement Learning with Confidence-Conditioned Verification), a novel confidence-adaptive framework that expands Pass@k coverage and improves Pass@1 performance. Based on our key insight that verification capability generally leads generation capability, TTRL-CoCoV employs a confidence-conditioned mechanism: for high-confidence samples, it bootstraps verifier and applies an exploration-enhancing reward to prevent diversity collapse; for low-confidence samples, it delegates pseudo-label selection to the verifier to filter incorrect pseudo-labels; and for medium-confidence samples, it bypasses verification entirely. Extensive experiments demonstrate that TTRL-CoCoV outperforms the best competing methods across 6 widely-recognized benchmarks, achieves average absolute gains of +9.8% in Pass@1 and +18.7% in Pass@16 over TTRL, and even achieves absolute Pass@1 improvements of up to +5.0% across multiple reasoning benchmarks when compared against fully supervised RL methods. Our code repository: https://github.com/shanjf666/CoCoV.