Organizations: Key Laboratory of Network Data Science and Technology, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, China · University of Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China · Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · University of California at Merced · University of Queensland
Reliable AI systems require large language models (LLMs) to exhibit behaviors aligned with human preferences and values. However, most existing alignment approaches operate at training time and rely on additional high-quality data, incurring significant computational and annotation costs. While recent work has shown that contrastive decoding can leverage a model's internal distributions to improve specific capabilities, its applicability remains limited to narrow behavioral scopes and scenarios. In this work, we introduce Polarity-Prompt Contrastive Decoding (PromptCD), a test-time behavior control method that generalizes contrastive decoding to broader enhancement settings. PromptCD constructs paired positive and negative guiding prompts for a target behavior and contrasts model responses-specifically token-level probability distributions in LLMs and visual attention patterns in VLMs-to reinforce desirable outcomes. This formulation extends contrastive decoding to a wide range of enhancement objectives and is applicable to both LLMs and Vision-Language Models (VLMs) without additional training. For LLMs, experiments on the "3H" alignment objectives (helpfulness, honesty, and harmlessness) demonstrate consistent and substantial improvements, indicating that post-trained models can achieve meaningful self-enhancement purely at test time. For VLMs, we further analyze contrastive effects on visual attention, showing that PromptCD significantly improves VQA performance by reinforcing behavior-consistent visual grounding. Collectively, these results highlight PromptCD as a simple, general, and cost-efficient strategy for reliable behavior control across modalities.
Figures & tables
Fig. 1: Overview of PromptCD. A unified framework for test-time behavior enhancement. By contrasting the logits (for text) or visual attention maps (for images) induced by positive and negative prompts, PromptCD effectively steers models toward desired goals without retraining. We demonstrate this capability through faithfulness enhancement in LLMs and visual attentiveness improvement in VLMs.
Fig. 2: Comparison of contextual and parametric knowledge shifts under positive and negative prompts. The first discriminative tokens are identified to represent the corresponding knowledge, and their logit and rank distributions are tracked to illustrate the underlying distributional dynamics.
Fig. 3: Selected cases showing changes in the first token for both parametric and contextual knowledge. The cases are obtained by introducing positive prompts carrying counterfactual information into the LLaMA2-7B-chat model. ’ → ’ denotes the knowledge shift after incorporating positive prompts. ’Logits’ and ‘rank’ refer to the first discriminative token in the knowledge answer, indicating the model’s knowledge confidence.
Fig. 4: Illustration of PromptCD enhancing context-faithfulness. By contrasting the probability distributions induced by positive and negative prompts, PromptCD amplifies contextual signals, guiding the LLM to generate faithful responses to the input question.
Fig. 5: The PromptCD framework for cross-modal attention refinement. By contrasting the task-specific attention distribution A(P) (derived from positive prompts) against the general attention pattern A(N) (derived from negative prompts), the method computes a refined attention map A^ . This contrastive mechanism effectively isolates and highlights request-relevant visual regions, thereby enhancing visual grounding.
Model
Method
NQ
ConFiQA
CoConflictQA
ConR (↑)
ParR (↓)
MR (↓)
ConR (↑)
ParR (↓)
MR (↓)
ConR (↑)
ParR (↓)
MR (↓)
LLaMA2-7B-chat
Vanilla
43.35
43.75
50.24
69.66
28.14
28.78
62.07
18.76
23.20
Attr
64.39
21.99
25.46
82.03
18.76
18.61
65.46
13.97
17.58
Opin
74.59
15.39
17.11
88.62
15.38
14.79
67.26
13.57
16.79
PromptCD
75.99
6.39
7.76
90.42
13.76
13.21
70.25
11.16
13.71
LLaMA3-8B-instruct
Vanilla
43.92
34.05
43.47
54.23
22.39
29.22
68.46
29.74
30.29
TABLE I: Performance (%) of PromptCD on LLM helpfulness evaluation.
Model
Method
TruthfulQA(MC)
FActScore
MC1
MC2
MC3
Score
#facts
Vanilla
33.66
51.29
24.91
33.20
25.60
LLaMA2-
DoLa
32.68
61.12
29.86
35.60
54.10
7B-chat
SLED
37.08
63.83
32.92
36.00
32.40
RECITE
32.19
50.38
25.06
36.10
32.40
PromptCD
42.59
64.05
33.89
36.60
46.40
TABLE II: Performance (%) of PromptCD on LLM honesty evaluation.
Method
Mode
Defense
Defense Generalization
Fluency
Success
onlyQ
otherA
otherQ
otherAQ
Vanilla
40.59
76.96
22.41
41.33
21.78
7.01
LLaMA2-
FT-L
42.52
76.22
28.37
42.89
28.85
5.94
7B-chat
DINM
90.00
89.93
59.26
87.41
57.63
6.61
PromptCD
96.67
98.67
82.30
96.59
82.37
7.29
Vanilla
12.15
54.81
18.07
12.07
18.74
7.46
TABLE III: Performance (%) of PromptCD on LLM harmlessness evaluation.
Fig. 6: Average performance across the three “3H” dimensions on different models. PromptCD consistently yields substantial improvements over the original models, while the baselines (Opin, DoLa, and DINM) show relatively limited gains.
Model
Layers L
A-OKVQA
POPE
V ∗
TextVQA
Qwen2.5-VL-3B
Vanilla
73.0
86.9
50.3
72.8
[10, 15]
74.0
86.9
53.4
73.0
[15, 20]
76.8
87.7
56.0
76.0
[20, 25]
78.3
87.9
57.1
76.3
Qwen2.5-VL-7B
Vanilla
75.0
87.0
50.8
75.0
[10, 15]
75.0
87.0
51.3
75.0
TABLE IV: Performance (%) of PromptCD on VLMs across different layer intervals.
Model
γ=0.2
γ=0.5
γ=0.8
Helpfulness (ConR)
77.32
78.89
70.53
Honesty (MC-AVG)
39.59
46.84
44.87
Harmlessness (DG-AVG)
83.29
89.98
85.34
TABLE V: Ablation results of the adjustment coefficient γ on LLaMA2-7B-chat models across the ‘3H’ dimensions. Performance is reported using the average values of ConR, MC-AVG, and DG-AVG, respectively.
Fig. 7: Visualization of contrastive attention refinement in VQA. As the threshold τ decreases from 0.6 to 0.1, PromptCD progressively suppresses visual noise to isolate request-relevant regions. Our PromptCD precise grounding corrects initial hallucinations (Red × ) and yields accurate answers (Green ✓ ).
Method
Original
SAM
YOLO
CLIP
ViCrop
PromptCD
rel-att
grad-att
pure-grad
Accuracy
47.80
49.42
48.84
48.55
55.17
56.06
51.67
58.2
GPU Time
0.17
3.33
0.35
1.07
1.16
0.89
2.36
1.34
TABLE VI: Performance comparison of PromptCD against external tool-based approaches and ViCrop on TextVQA dataset: accuracy (%) and inference time (seconds) overhead per sample.
Model
Method
Latency (ms/token)
Throughput (token/s)
LLaMA2-7B
Vanilla
36.03 ( × 1.00)
27.76 ( × 1.00)
PromptCD
64.85 ( × 1.79)
16.64 ( × 0.60)
LLaMA2-13B
Vanilla
51.41 ( × 1.00)
19.45 ( × 1.00)
PromptCD
89.45 ( × 1.74)
12.25 ( × 0.63)
LLaMA3-8B
Vanilla
42.17 ( × 1.00)
23.92 ( × 1.00)
PromptCD
69.77 ( × 1.65)
14.08 ( × 0.59)
TABLE VII: Comparison of inference latency and throughput for PromptCD.
Model
Vanilla
PromptCD
w/ APC
w/o APC
LLaMA2-7B
78.9
77.8
30.7
LLaMA3-8B
83.7
85.5
53.8
Mistralv0.3-7B
92.4
91.7
49.9
Qwen2.5-7B
88.8
89.8
62.4
TABLE VIII: Ablation study on the NQ dataset evaluating the effect of the Adaptive Plausibility Constraint (APC) on generation quality (Hit Rate).
Model
Method
MQuAKE
3k
2002
hard
LLaMA2-7B-chat
IKE
20.7
20.6
2.3
+ PromptCD
22.4
20.4
3.8
MeLLo
32.6
40.8
5.1
+ PromptCD
43.1
45.8
5.8
LLaMA2-13B-chat
IKE
19.4
18.8
2.7
TABLE IX: Experimental results (accuracy %) using LLaMA2-chat models. We conduct experiments with full-batch edit memory to evaluate the performance of memory-based Knowledge Editing.
Fig. 8: Performance comparison (%) between standard IKE and IKE enhanced with PromptCD across various models and datasets. We set the edit memory batch size to 1 to evaluate the fundamental capability of direct knowledge editing.
Fig. 9: Probability statistics of edited new knowledge on the MQuAKE-Stubborn dataset using LLaMA2-7B-chat . The probabilities are computed by applying softmax to the model’s token logits.
Inference-time language model alignment steers or selects outputs during generation to better meet safety, preference, and task goals without retraining.
Vision-Language-Action (VLA) models trained on large-scale data have made remarkable progress, but they remain vulnerable to distribution shifts at deployment time. Recent VLA models suggest that prompts can serve as an efficient interface for steering policy behavior, but existing prompt-based steering typically relies on external guidance. This raises a natural question: can test-time training (TTT) for VLA be achieved by optimizing a prompt, so that the steering interface itself can be learned and adapted from interaction? We address this question with TTT-VLA, a test-time training framework based on Latent Prompt Optimization (LPO). During training, the latent prompt is learned with an additional proxy task, providing an extra learned conditioning signal for policy learning. At test time, TTT is performed by collecting interaction data from the current environment and optimizing only the latent prompt on those data using the proxy task's self-supervised signal, without modifying the policy itself. Experiments on SimplerEnv demonstrate that the proposed method consistently improves task success rates in both single- and multi-embodiment settings. Further analysis shows that the gains arise primarily from correcting a small number of critical decisions rather than globally altering policy behavior. These results suggest that LPO provides an effective and practical pathway for deployment-time improvement of foundation manipulation policies.
Wenbo Zhang, Jianxiong Li, Shuai Yang +4
ByteDance Seed · The University of Adelaide · Tsinghua University +3
Large language models (LLMs) are increasingly used as surrogates for human participants, but it remains unclear which models best capture human behavior and why. To address this, we introduce Psych-201, a novel dataset that enables us to measure behavioral alignment at scale. We find that post-training -- the stage that turns base models into useful assistants -- consistently reduces alignment with human behavior across model families, sizes, and objectives. Moreover, this misalignment widens in newer model generations even as base models continue to improve. Finally, we find that persona-induction -- a popular technique for eliciting human-like behavior by conditioning models on participant-specific information -- does not improve predictions at the level of individuals. Taken together, our results suggest that the very processes that are currently employed to turn LLMs into useful assistants also make them less accurate models of human behavior.
Alignment in large language models (LLMs) is still largely static: after training, the policy is frozen. DPO, GRPO methods typically imprint one behavior into the weights, leaving little runtime control beyond prompt hacks or expensive re-alignment. We introduce ECLIPTICA, which treats alignment as instruction-driven and runtime-controllable: natural-language alignment instructions act as an explicit behavioral contract (stance, refusal boundary, verbosity) that modulates behavior on the fly under evolving safety requirements, user roles, and governance constraints. We introduce CITA (Contrastive Instruction-Tuned Alignment), combining SFT with contrastive preference optimization under an explicit geometric anchor to a reference model. This yields a stable Riemannian chart and keeps instruction updates within a shared neighborhood, so regimes stay nearby and traversable for reliable switching. To isolate policy switching from ordinary instruction following, we release the ECLIPTICA benchmark: 3000 controlled cases (300 prompts x 10 instruction types) where the user request is fixed and only the alignment instruction changes. On Llama-3.1-8B across five suites (ECLIPTICA, TruthfulQA, Conditional Safety, Length Control, LITMUS), CITA reaches 86.7% instruction-alignment efficiency, beating DPO (56.1%), GRPO (36.1%), and PPO (20.4%).
Kapil Wanaskar, Gaytri Jena, Vinija Jain +2
San José de State University, USA · Pragya Lab, BITS Pilani Goa, India · Google, USA +1