Training-Free Test-Time Contrastive Learning for Large Language Models

Kaiwen Zheng; Kai Zhou; Jinwu Hu; Te Gu; Mingkai Peng; Fei Liu

cs.CLApr 15, 2026

Training-Free Test-Time Contrastive Learning for Large Language Models

Authors: Kaiwen Zheng, Kai Zhou, Jinwu Hu, Te Gu, Mingkai Peng, Fei Liu

Organizations: South China University of Technology · Pazhou Laboratory

Abstract

Large language models (LLMs) demonstrate strong reasoning capabilities, but their performance often degrades under distribution shift. Existing test-time adaptation (TTA) methods rely on gradient-based updates that require white-box access and need substantial overhead, while training-free alternatives are either static or depend on external guidance. In this paper, we propose Training-Free Test-Time Contrastive Learning TF-TTCL, a training-free adaptation framework that enables a frozen LLM to improve online by distilling supervision from its own inference experiences. Specifically, TF-TTCL implements a dynamic "Explore-Reflect-Steer" loop through three core modules: 1) Semantic Query Augmentation first diversifies problem views via multi-agent role-playing to generate different reasoning trajectories; 2) Contrastive Experience Distillation then captures the semantic gap between superior and inferior trajectories, distilling them into explicit textual rules; and 3) Contextual Rule Retrieval finally activates these stored rules during inference to dynamically steer the frozen LLM toward robust reasoning patterns while avoiding observed errors. Extensive experiments on closed-ended reasoning tasks and open-ended evaluation tasks demonstrate that TF-TTCL consistently outperforms strong zero-shot baselines and representative TTA methods under online evaluation. Code is available at https://github.com/KevinSCUTer/TF-TTCL.

Figures & tables

Paradigms	External Knowledge	Source Data	Gradient-Free	Online
Retrieval Augmentation Generation Lewis et al. (2020)	✓	✗	✓	✓
Test-Time Adaptation Wang et al. (2021)	✗	✗	✗	✓
Test-Time Training Hardt and Sun (2024)	✓	✓	✗	✓
Test-Time Reinforcement Learning Zuo et al. (2025)	✗	✗	✗	✗
Test-Time Learning Hu et al. (2025a)	✗	✗	✗	✓
Reasoning-Bank Ouyang et al. (2025)	✓	✗	✗	✓

Table 1: Comparison of different test-time paradigms. TF-TTCL (Ours) is a gradient-free adaptation framework capable of online evaluation, requiring neither source data nor an external knowledge.

Figure 1: Overview of the TF-TTCL framework. 1) Semantic Query Augmentation: Employs multi-agent role-playing to probe diverse reasoning trajectories. 2) Contrastive Experience Distillation: Distills the semantic gap between selected positive and negative samples into textual rules for memory. 3) Contextual Rule Retrieval: Retrieves relevant historical insights from the rule repository to guide the inference.

Figure 2: The pipeline of Contrastive Experience Distillation. Our approach partitions outputs into Positive and Negative Candidates using consistency-based candidate partitioning. We select the positive and negative candidates via min-PPL selection. The final adaptation rules are generated by summarizing the reasoning gap.

Panel of Figure 3 — Figure 3: A representative example demonstrating the extraction of useful contrastive rules ( $r^{+},r^{-}$ ) from reasoning gaps, which serve as explicit guidance for subsequent problem-solving steps.

Method	Publication	GSM8k	MATH-500	AIME24	Minerva	Average
Base LLM	-	82.49	49.20	3.33	20.96	39.00
Tent Wang et al. (2021)	ICLR 2021	70.20	49.20	10.00	21.32	37.68
EATA Niu et al. (2022)	ICML 2022	75.06	49.40	6.67	20.96	38.02
COME Zhang et al. (2025b)	ICLR 2025	75.59	48.80	6.67	20.96	38.01
TLM Hu et al. (2025a)	ICML 2025	85.06	50.00	6.67	19.49	40.31
TF-GRPO ( Cai et al., 2025 )	arXiv 2025	86.49	53.00	3.33	21.69	41.13

Table 2: Comparison with SOTA methods on Closed-ended Reasoning Task using Llama-3.1-8B-Instruct. The metric is Accuracy (%). "Base LLM" denotes the zero-shot baseline. The best results are bolded .

Method	Publication	Geography	Agriculture	Medicine	Finance	Average
Base LLM	-	0.2441	0.0876	0.1356	0.2251	0.1731
Tent Wang et al. (2021)	ICLR 2021	0.2682	0.0624	0.1448	0.2140	0.1724
EATA Niu et al. (2022)	ICML 2022	0.2757	0.0626	0.1455	0.1886	0.1681
COME Zhang et al. (2025b)	ICLR 2025	0.2636	0.0407	0.1382	0.0699	0.1281
TLM Hu et al. (2025a)	ICML 2025	0.2620	0.0956	0.1372	0.2295	0.1811
TF-GRPO ( Cai et al., 2025 )	arXiv 2025	0.2260	0.0993	0.1147	0.2071	0.1618

Table 3: Comparison with SOTA methods on Open-ended Evaluation Task using Llama-3.1-8B-Instruct. The metric is ROUGE-Lsum (higher is better). "Base LLM" denotes the zero-shot baseline. The best results are bolded .

Method	Qwen-Plus		DeepSeek-V3.2
Method	AIME24	Finance	AIME24	Finance
Base LLM	30.00	0.2647	66.67	0.2578
Chain-of-Thought	40.00	0.2297	70.00	0.2428
TF-GRPO	70.00	0.2500	80.00	0.2580
TF-TTCL (Ours)	76.67	0.2831	83.33	0.2919

Table 4: Performance comparison on API-based Models. We compare our training-free approach against standard Chain-of-Thought Wei et al. (2022) and TF-GRPO Cai et al. (2025) on AIME24 (Reasoning) and Finance (Domain). The best results are bolded .

Method	GSM8k	Finance
Baseline	82.49	0.2251
TF-TTCL w/o SQA	87.11	0.2851
TF-TTCL w/o CED	85.97	0.2639
TF-TTCL w/o CRR	87.34	0.2596
TF-TTCL (Ours)	87.49	0.2863

Table 5: Full ablation study of TF-TTCL on GSM8k and Finance. The best results are bolded .

Component	Variant	GSM8k	Finance
CED	w/o positive rules	87.19	0.2812
	w/o negative rules	86.88	0.2668
	Ours	87.49	0.2863
CRR	w/ random rules	87.41	0.2665
CRR	Ours	87.49	0.2863

Table 6: Ablation study on key components of TF-TTCL. For CED, we remove either the positive-rule set or the negative-rule set to examine their individual effects. For CRR, we replace curated rules with randomly sampled rules. The best results are bolded .

Variants	Latency	Memory	Acc.
Single Call	2.05s	-	82.49
TF-TTCL (Sequence)	10.25s	Unbounded	87.49
TF-TTCL (Parallel)	4.11s	Unbounded	87.49
+ Pruning Strategy	4.11s	1,000	87.72

Table 7: System efficiency and scalability on GSM8k. Parallel execution caps latency, while memory pruning bounds repository growth and improves performance.

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Task / Description
DomainBench
Geography	Knowledge-intensive Q&A
Agriculture	Agricultural production Q&A
Medicine	Patient-Doctor Dialogue
Finance	Sentiment Analysis & Financial Q&A
Math Benchmarks (Test Sets)

Appendix

Table 8: Description of the eight evaluation datasets employed in our experiments, grouped into domain-specific question answering (DomainBench) and mathematical reasoning benchmarks (Math Benchmarks).

Figure 4: Hyper-parameter ablation for TF-TTCL. “Max #Rules =

K

” denotes

K

positive and

K

negative rules. “Sample Instances =

N

” denotes

N

Student instances. Results are reported on GSM8k (Accuracy) and Finance (ROUGE-Lsum).

Model	Zero-shot	CoT	TF-GRPO	TF-TTCL (Ours)	$\Delta_{abs}$	RER (%)
Llama-3.2-Instruct-3B	69.90	71.87	80.42	83.09	+13.19	43.8
Llama-3.1-Instruct-8B	82.49	85.82	86.49	87.49	+5.00	28.6
Qwen3-Instruct-32B	89.69	90.30	90.37	95.30	+5.61	54.4
Llama-3.3-Instruct-70B	90.14	86.96	90.14	95.07	+4.93	50.0
Qwen3-235B-A22B	89.16	89.08	89.76	95.45	+6.29	41.9

Appendix

Table 9: Performance scaling and Relative Error Reduction (RER) across a wide size spectrum (3B to 235B) on the GSM8k dataset.

Method	GSM8k	Finance
Zero-shot	69.90	0.2319
CoT	71.87	0.2206
TF-GRPO	80.42	0.1922
TF-TTCL (Ours)	83.09	0.2357

Appendix

Table 10: Robustness on weak backbone configurations (Llama-3.2-3B-Instruct) across reasoning and open-ended evaluation tasks.

Repo Size ( $\times$ 1k Rules)	Mean (s)	Median (s)	Std (s)
1	0.0055	0.0045	0.0036
5	0.0210	0.0198	0.0034
10	0.0787	0.0779	0.0036
50	0.4026	0.4256	0.0446
100	0.6173	0.6148	0.0071

Appendix

Table 11: Latency overhead across ascending rule repository scales (retrieval with Qwen3-0.6B-Embedding, averaged over 100 queries).

Dataset Setup	w/o SQA	Full TF-TTCL	$\Delta_{abs}$
GSM8k (CRT Easy)	87.11	87.49	+0.38
AIME (CRT Hard)	6.67	13.33	+6.66
Finance (OET Easy)	0.2851	0.2863	+0.0012
Medicine (OET Hard)	0.1784	0.2018	+0.0234

Appendix

Table 12: Performance gap when removing SQA alongside simple vs. hard datasets on Llama-3.1-8B-Instruct.

Configuration	Accuracy
Positive Rules Only	14.76
Negative Rules Only	15.13
Combined (Positive + Negative)	16.61

Appendix

Table 13: Performance on the MATH-500-3B-Wrong subset when relying on different types of extracted rules.

Method Variant	GSM8k	Finance
Zero-Shot	82.49	0.1731
ReasoningBank (LLM-as-Judge)	82.64	0.2752
TF-TTCL (Ours)	87.49	0.2863

Appendix

Table 14: Comparison with an LLM-as-Judge partitioning strategy (ReasoningBank approach) lacking fully unsupervised capability, using Llama-3.1-8B.

Selection Metric	GSM8k	Finance
min-Entropy	87.04	0.2458
min-Perplexity (Ours)	87.49	0.2863

Appendix

Table 15: Ablation of distinct statistical filtering criteria on candidate solutions.

Method	BERTScore $\uparrow$	BLEURT $\uparrow$	BLEU $\uparrow$	Rouge-1 $\uparrow$	Rouge-2 $\uparrow$	Rouge-L $\uparrow$
Geography	0.6909	-0.6800	0.0685	0.2701	0.1025	0.1955
$\bullet$ Tent	0.6966	-0.7273	0.0857	0.2959	0.1269	0.2368
$\bullet$ EATA	0.7033	-0.6088	0.0870	0.3039	0.1243	0.2332
$\bullet$ COME	0.6985	-0.6298	0.0790	0.2900	0.1158	0.2161
$\bullet$ TLM	0.6980	-0.6772	0.0785	0.2903	0.1167	0.2147
$\bullet$ TF-GRPO	0.6717	-0.7330	0.0544	0.2487	0.0948	0.1794

Appendix

Table 16: Performance on DomainBench across four domains: Geography, Agriculture, Finance, and Medicine. The best and second-best results are highlighted in bold and underlined , respectively.

Method	BERTScore $\uparrow$	BLEURT $\uparrow$	BLEU $\uparrow$	Rouge-1 $\uparrow$	Rouge-2 $\uparrow$	Rouge-L $\uparrow$
Qwen-Plus	0.7156	-0.5035	0.0751	0.2936	0.1155	0.2257
$\bullet$ Chain of Thoughts	0.7002	-0.5858	0.0462	0.2580	0.0858	0.1877
$\bullet$ TF-GRPO	0.6818	-0.5992	0.0441	0.2769	0.0862	0.1799
$\bullet$ TF-TTCL(Ours)	0.7164	-0.5110	0.0777	0.3169	0.1154	0.2353
Deepseek-V3.2	0.7163	-0.5799	0.0734	0.2831	0.1108	0.2271
$\bullet$ Chain of Thoughts	0.7115	-0.5624	0.0616	0.2705	0.1006	0.2106

Appendix

Table 17: Performance of API models on the Finance subset of DomainBench. The best and second-best results are highlighted in bold and underlined , respectively.

Component Ablation
Method	BERTScore $\uparrow$	BLEURT $\uparrow$	BLEU $\uparrow$	Rouge-1 $\uparrow$	Rouge-2 $\uparrow$	Rouge-L $\uparrow$
Llama-3.1-8B-Instruct	0.6806	-0.6517	0.0372	0.2448	0.0804	0.1615
$\bullet$ w/o SQA	0.7121	-0.5234	0.0751	0.3148	0.1228	0.2375
$\bullet$ w/o CED	0.6917	-0.5898	0.0537	0.2909	0.0964	0.1932
$\bullet$ w/o CRR	0.6975	-0.5297	0.0602	0.2869	0.1051	0.2042
Rule Design Ablation

Appendix

Table 18: Comprehensive ablation study of TF-TTCL on the Finance subset of DomainBench, analyzing the impact of key components and rule designs. The best and second-best results across all variants are highlighted in bold and underlined , respectively.

Figure 5: Schematic diagram of the problem

Figure 6: General Teacher System Prompt for CRT.

Figure 7: General Student System Prompt for CRT.

Figure 8: General Output Format for CRT.

Figure 9: General Rules Injection Prompt.

Figure 10: General Tutor System Prompt for CRT.

Figure 11: General Positive Rules Summarization System Prompt for CRT.

Figure 12: General Negative Rules Summarization System Prompt for CRT.

Figure 13: General Teacher System Prompt for OET.

Figure 14: General Student System Prompt for OET.

Figure 15: General Tutor System Prompt for OET.

Figure 16: General Positive Rules Summarization System Prompt for OET.

Figure 17: General Negative Rules Summarization System Prompt for OET.

Figure 18: Teacher System Prompt for GSM8k.

Figure 19: Student System Prompt for GSM8k.

Figure 20: Unified Format Prompt for GSM8k.

Figure 21: Tutor System Prompt for GSM8k.

Figure 22: Positive Rules Summarization System Prompt for GSM8k.

Figure 23: Negative Rules Summarization System Prompt for GSM8k.

Figure 24: Teacher System Prompt for Finance.

Figure 25: Student System Prompt for Finance.

Figure 26: Tutor System Prompt for Finance.

Figure 27: Positive Rules Summarization System Prompt for Finance.

Figure 28: Negative Rules Summarization System Prompt for Finance.

Test-Time Adaptation

Test-time adaptation updates or recalibrates pretrained models at inference to handle distribution shifts, often without target labels.

See all 322 papers

Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning
Sep 10, 2026Bincheng Gu, Min Gao, Zongwei Wang +3Test-Time Adaptation LLM Reliability
Key Laboratory of Dependable Service Computing in Cyber Physical Society, Chongqing University, China · King’s College London, London, UK · Griffith University, Brisbane, Australia
TARo: Token-level Adaptive Routing for LLM Test-time Alignment
Mar 19, 2026Arushi Rai, Qiang Zhang, Hanqing Zeng +5Inference-Time Language Model Alignment LLM Reasoning
Meta · University of Pittsburgh · University of California, Berkeley
Query-Conditioned Test-Time Self-Training for Large Language Models
May 13, 2026Chaehee Song, Minseok Seo, Yeeun Seong +2Test-Time Adaptation Test-Time Optimization
School of Electrical Engineering, KAIST · 2Graduate School of Green Growth and Sustainability, KAIST

Sep 10, 2026J/K move · Enter open · S save

Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning

Bincheng Gu, Min Gao, Zongwei Wang, Yibing Bai+2

Key Laboratory of Dependable Service Computing in Cyber Physical Society, Chongqing University, China · King’s College London, London, UK · Griffith University, Brisbane, Australia

Test-Time Adaptation LLM Reliability Inference-Time Adaptation of Language Models LLM Reasoning

Open paper AskPDF arXiv

Test-time adaptation has emerged as a lightweight alternative to costly post-training for improving the reasoning capabilities of Large Language Models (LLMs) on downstream tasks. Predictive entropy provides a model-derived signal for such adaptation, guiding models toward higher-confidence reasoning states without external verifiers or reward models. However, higher confidence does not necessarily imply correctness, as LLMs may remain highly confident along incorrect reasoning trajectories. We observe that high-confidence reasoning is more likely to be correct when confidence remains stable under local perturbations. Based on this observation, we propose Test-Time Adaptation via Stability-Aware Confidence Optimization (TASCO), a framework that incorporates local stability into confidence-based test-time adaptation while keeping the LLM frozen. TASCO operationalizes local stability by optimizing a lightweight task-level prefix under two alternative perturbation strategies: Random Perturbation promotes distributional stability across trajectories induced by nearby perturbed prefixes, whereas Sharpness-Aware Perturbation targets worst-case local sensitivity. Experiments demonstrate that TASCO improves reasoning accuracy and token efficiency across diverse LLMs and reasoning benchmarks, while behavioral analyses show that it maintains stable confidence under local perturbations without prematurely concentrating the model's predictive distribution.

Abstract

Test-Time Adaptation

Explore similar work

Beyond Confidence: Stability-Aware Test-Time Adaptation for LLM Reasoning

TARo: Token-level Adaptive Routing for LLM Test-time Alignment

Query-Conditioned Test-Time Self-Training for Large Language Models