Test-Time RL

RL: Reinforcement Learning

Test-time reinforcement learning studies how models and agents can improve their decisions on encountered problems using feedback available during inference, rather than relying only on a conventional labeled training set. For language models, common approaches use majority-vote pseudo-labels, confidence or external verification, process rewards, and policy-gradient updates; other systems retrieve experience or steer frozen policies with value gradients, search, or execution feedback. These methods are applied to reasoning, code generation, web agents, multimodal tasks, and robot control, drawing feedback from test cases, tools, simulators, or interaction. A central challenge is that self-generated rewards can reinforce wrong consensus, suppress rare correct answers, or destabilize behavior, making reliable verification, exploration, and compute-efficient gains key open concerns.

Latest papers

48 papers10 papers in the last four weeks, up 67% on the four weeks before. 0.1% of all new papers.

MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning
Oct 23, 2026Zhixia Zhang, Binqian Xu, Huaiyang Wang +2Test-Time RL RL for Language Model Reasoning
Beihang University · National University of Singapore
METTA: Memory-Enhanced Test-Time Adaptation for Meta-Reinforcement Learning
Oct 22, 2026Ali Najar, Hosein Hasani, Amir Kooshan Fattah +2Meta-Reinforcement Learning Test-Time RL
Department of Computer Engineering Sharif University of Technology
Continual Test-Time Learning with Evolution Strategies
Oct 16, 2026Guangya Hao, Yiting Duan, Longyan Tan +5Continual Learning for LLMs Continual Test-Time Adaptation
University of Cambridge · Western Sydney University · The Alan Turing Institute +1
Efficient Test-Time Adaptation For Robot Policies
Oct 14, 2026Motasem Alfarra, Pietro Mazzaglia, Markus Peschl +2Robot Policy Adaptation Test-Time RL
Qualcomm AI Research, Amsterdam, the Netherlands
SelectTTRL: Representation-Guided Data Selection for Efficient Test-Time Reinforcement Learning
Oct 12, 2026Yi Li, Linan Yue, Weibo GaoTest-Time RL RL for Language Model Reasoning
Southeast University · The Hong Kong Polytechnic University
Evolving in Thought Space: Training a Small Model at Test Time Unlocks Better Discoveries
Oct 5, 2026Chonghe Jiang, Ao Qu, Siyuan Liu +9Large Language Model-Guided Optimization Test-Time RL
Massachusetts Institute of Technology · Singapore-MIT Alliance for Research and Technology · Hong Kong Polytechnic University +6
CURIO: Curiosity-Driven Test-Time Learning for Open-Ended Discovery
Oct 4, 2026Tao Feng, Fangxu Yu, Zijie Lei +5Test-Time RL RL for Language Models
University of Illinois Urbana-Champaign · University of Maryland, College Park · Stanford University
Test-time Multi-agent Coordination by Decomposed Value Gradient Flow
Oct 1, 2026Dongsu Lee, Haoran Xu, Amy ZhangMulti-Agent Coordination Offline MARL
The University of Texas at Austin
InterEvolve: Test-Time Evolution of Reward Programs for Humanoid Loco-Manipulation
Oct 1, 2026Zhuo Lin, Sirui Xu, Liuyu Bian +2Humanoid Loco-Manipulation Reinforcement Learning
University of Illinois Urbana-Champaign
Sharpen Before You Adapt: Data-Free Entry-State Sharpening for Test-Time Reinforcement Learning
Oct 1, 2026Zhanming Zhang, Vinoth SelvendranRL for Language Model Reasoning Test-Time RL
Independent Researcher New York, NY · Independent Researcher Palo Alto, CA
Noisy Test-Time Reinforcement Learning for Code LLMs
Sep 26, 2026Xikai Yang, Hieu Trung Nguyen, Dunyuan Xu +4Language Model Robustness RL for Code Generation
The Chinese University of Hong Kong · Huazhong University of Science and Technology
G2MAF: Test-Time Gradient Guidance for Multi-Agent Flow Policies
Sep 25, 2026Guowei Zou, Haitao Wang, Guoxin Wang +4Cooperative MARL Test-Time RL
Sun Yat-sen University
Label-free steering: Compressing test-time reinforcement learning into bias-only subspaces
Sep 16, 2026Naveen Vakada, Mingyuan Li, Shaoxiong JiFine-Tuning Test-Time Adaptation of VLMs
University of Turku, Finland · ELLIS Institute Finland
Rewarding Reasoning, Not Answers: Fixing and Bounding Test-Time Reinforcement Learning on Medical QA
Sep 15, 2026Kailong Fan, Anqi Pu, Yichen Wu +7Process Reward Models RL for Language Model Reasoning
1Harvard Medical School/MGH · 3Harvard University · 2Zhejiang University
Entropy-Regularized Rank-Masked Policy Optimization for Test-Time Reinforcement Learning in Code Generation
Sep 8, 2026Jiacheng Xu, Feng Chen, Xiuneng Xu +1RL for Code Generation Policy Optimization
Nanyang Technological University, Singapore
APEx: Distillation of Agent Procedural Experience for Adaptive Deep Research Question Answering
Sep 2, 2026Jie Ding, Rui Sun, Xinyuan Zhang +2LLM Agents Agent Memory
University of Science and Technology of China · 3ByteDance Inc. · 2Chinese Academy of Sciences
TTPO: Test-Time Policy Optimization
Aug 27, 2026Aozhe Wang, Zhengxi Lu, Jianze Wang +8Test-Time Training Test-Time RL
Zhejiang University · Alibaba Group
Test-time Reinforcement Learning for Anomalous Video Understanding
Aug 18, 2026Huining Li, Yuxiang Duan, Jiyang Tan +5Test-Time RL Video Anomaly Detection
C-FAIR & School of software, Shandong University · Nanjing University of Posts and Telecommunications · Lenovo Research
CoRE: Consensus Rewards via Equilibrium for Test-Time Reinforcement Learning
Aug 10, 2026Ambuj Mehrish, Sebastiano VasconReward Modeling RL for Language Models
CVML Lab Ca’ Foscari University of Venice
Hi-TTRL: Regulating Consensus with Hints for Test-Time Reinforcement Learning
Aug 4, 2026Kunbin Xu, Xingzuo Li, Xuefeng Bai +1RL for Language Model Reasoning Test-Time RL
School of Computer Science and Technology, Harbin Institute of Technology, Shenzhen, China
SERPO: Self-Evolving Rubric Policy Optimization for Open-Ended Test-Time Reinforcement Learning
Jul 29, 2026Jianze Wang, Kunwang Zheng, Ying Liu +5Reinforcement Learning Test-Time RL
School of Artificial Intelligence and Automation, Huazhong University of Science and Technology · Alibaba Group · University of Science and Technology of China
AdaTurn: Budget-Aware Test-Time Scaling for Active Visual Perception Agents
Jul 16, 2026Susan Liang, Chao Huang, Filippos Bellos +3Test-Time Scaling Tool-Using Vision-Language Agents
University of Rochester · Univeristy of Michigan
ELASTIC: Efficiently Learning to Adaptively Scale Test-Time Compute for Generative Control Policies
Jun 30, 2026Andrew Zou Li, Gokul Swamy, Yonatan Bisk +1Diffusion Policy Test-Time RL
Carnegie Mellon University
Trust Your Instincts: Confidence-Driven Test-Time RL for Vision-Language-Action Models
Jun 29, 2026Siyao Chen, Jiakang Yuan, Jiaxin Wang +1Intrinsic Reward Methods Reinforcement Learning
College of Future Information Technology, Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · MoShen Intelligence, Shanghai, China
ReGuide: From Test-Time Guidance to Self-Improving Diffusion Policies
Jun 27, 2026Tzu-Hsiang Lin, Srinivas Shakkottai, Dileep Kalathil +1Robot Policy Adaptation Robot Failure Recovery
Texas A&M University
StarOR: Synergizing Tree Search and Test-Time Reinforcement Learning for Optimization Modeling
Jun 13, 2026Jiajun Li, Yu Ding, Shisi Guan +2Reinforcement Learning Large Language Model-Guided Optimization
School of Computer Science and Engineering, Southeast University · Northwest A&F University
QPILOTS: Efficient Test-Time Q-Steering for Flow Policies
Jun 11, 2026Yifan Ruan, Chenyang Cao, Andreas Burger +7Test-Time RL Offline-To-Online RL
University of Toronto · Vector Institute · LG Electronics
From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory
Jun 7, 2026Yishuo Cai, Xingyu Guo, Xuancheng Huang +8Continual Learning for LLM Agents LLM Agent Memory
Peking University, Beijing, China · Central South University, Changsha, China · Zhipu AI, Beijing, China +3
Agentic Monte Carlo: Simulating Reinforcement Learning for Black-Box Agents
Jun 3, 2026Dae Yon Hwang, Raunaq Suri, Valentin Villecroze +4Reinforcement Learning Bayesian RL
Layer 6 AI, Toronto, Canada.
Alpha-RTL: Test-Time Training for RTL Hardware Optimization
Jun 3, 2026Peilong Zhou, Zhirong Chen, Cangyuan Li +4RL for Language Models Test-Time Optimization
1SKLP, Institute of Computing Technology, Chinese Academy of Sciences · University of the Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences

Oct 23, 2026J/K move · Enter open · S save

MS-TTRL: Breaking the Frequency Conflict in Multiple-Solution Test-Time Reinforcement Learning

Zhixia Zhang, Binqian Xu, Huaiyang Wang, Yikun Ban+1

Beihang University · National University of Singapore

Test-Time RL RL for Language Model Reasoning Reinforcement Learning with Verifiable Rewards Behavioral Diversity in RL

Open paper AskPDF arXiv

Scientific, engineering, and mathematical reasoning tasks can admit multiple valid solutions, while exhaustive annotation of the full solution set is often prohibitively costly. Test-time reinforcement learning (TTRL) applies reinforcement learning to unlabeled test instances, offering a label-free approach to multiple-solution reasoning. In this setting, generation frequency serves two opposing objectives: correctness estimation favors frequent outcomes, whereas outcome-level diversity preservation favors less frequent outcomes. We term this tension the Frequency Conflict. To resolve this conflict, we propose MS-TTRL, which obtains frequency-independent correctness estimates through Outcome-Level Self-Verification while reserving generation frequency solely for diversity preservation. Three-State Outcome Cache and the Candidate State support verification reuse across repeated observations and confidence-aware handling of ambiguous outcomes, while Confidence-Gated Inverse Probability Scaling selectively strengthens the learning signals of rare, reliably verified outcomes. Experiments across two benchmarks, four multiple-solution tasks, and three backbone models, each evaluated with five random seeds, show that MS-TTRL improves Recovery Rate by 19.13 percentage points on average over the best-performing label-free baseline and approaches the performance of the supervised IPS-GRPO.