Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry

Guoxi Zhang; Jiawei Chen; Tianzhuo Yang; Lang Qin; Juntao Dai; Yaodong Yang; Jingwei Yi

cs.LGMar 27, 2026

Stable Reasoning, Unstable Responses: Mitigating LLM Deception via Stability Asymmetry

Authors: Guoxi Zhang, Jiawei Chen, Tianzhuo Yang, Lang Qin, Juntao Dai, Yaodong Yang, Jingwei Yi

Organizations: Institute for Artificial Intelligence, Peking University · Beijing Academy of Artificial Intelligence · School of Chinese as a Second Language, Peking University

Abstract

As Large Language Models (LLMs) expand in capability and application scope, their trustworthiness becomes critical. A vital risk is intrinsic deception, wherein models strategically mislead users to achieve their own objectives. Existing alignment approaches based on chain-of-thought (CoT) monitoring supervise explicit reasoning traces. However, under optimization pressure, models are incentivized to conceal deceptive reasoning, rendering semantic supervision fundamentally unreliable. Grounded in cognitive psychology, we hypothesize that a deceptive LLM maintains a stable internal belief in its CoT while its external response remains fragile under perturbation. We term this phenomenon stability asymmetry and quantify it by measuring the contrast between internal CoT stability and external response stability under perturbation. Building on this structural signature, we propose the Stability Asymmetry Regularization (SAR), a novel alignment objective that penalizes this distributional asymmetry during reinforcement learning. Unlike CoT monitoring, SAR targets the statistical structure of model outputs, rendering it robust to semantic concealment. Extensive experiments confirm that stability asymmetry reliably identifies deceptive behavior, and that SAR effectively suppresses intrinsic deception without degrading general model capability.

Figures & tables

Figure 1: Conceptual illustration of Stability Asymmetry . A deceiver maintains a consistent internal belief while providing a conflicting external response, leading to cue leakage under perturbation.

Figure 2: The Stability Asymmetry Regularization Framework. (a) Conceptual landscape illustrating the SAR Cost mechanism, where deceptive behaviors (high belief stability but low behavior stability) receive heavy penalties to push the policy into the safe zone. (b) The overall pipeline, detailing how perturbations are used to compute external and internal stabilities ( $S_{\text{ext}}$ and $S_{\text{int}}$ ), which are then fused via a soft gate to calculate the SAR Cost for Lagrangian-based policy updates.

Figure 3: Comparison of four stability metrics across two deception scenarios and two base models. SE demonstrates the most stable and consistent separability for both CoT and Response.

Figure 4: Visualization of three behavioral modes in the two-dimensional stability space using SE. Deceptive samples (red) exhibit the characteristic upper-left positioning, confirming the predicted stability asymmetry.

Strategic Deception
Method	Deception (%)			General Capability $\uparrow$
	CoT Plan.	Actual Decep. $\downarrow$	CoT Faith. $\uparrow$	GSM8K	IFEval	MMLU	TruthfulQA	Toxigen
Llama-3.1-8B	-	-	-	75.82	49.36	68.12	54.05	53.30
with GRPO	28.86	53.51	52.60	75.82	45.72	65.24	56.42	53.51
with Honesty Prompt	25.40	49.30	50.90	75.82	45.72	65.24	56.42	53.51
with CoT Monitor	0.20	86.57	13.42	75.06	45.11	66.19	57.98	55.53

Table 1: Deception metrics and general capability across methods and scenarios.

Figure 5: A case study of intrinsic deception mitigation, showing that GRPO produces deceptive outputs, CoT Monitor obfuscates deceptive intent, and SAR successfully enforces honest reasoning.

Figure 6: Performance comparison of ablation variants. Our method attains the best safety-capability trade-off with the highest composite score (

S=74

). Composite score details are in Appendix B.2.7 .

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

LoRA Configuration
Parameter	Value
LoRA Rank ( $r$ )	128
LoRA Alpha ( $\alpha$ )	256
LoRA Dropout	0.05
Training Configuration (GRPO)
Learning Rate	$1\times 10^{-5}$

Appendix

Table 2: Hyperparameters for GRPO Training with LoRA.

Parameter	Value	Description
$\epsilon_{\text{cost}}$	0.3	Cost threshold: lower values cause $\lambda$ to grow more often.
$T_{\text{gate}}$	2.0	Gate temperature: higher values slow gate decay.
$P_{\text{fmt}}$	$-3.0$	Penalty for missing <think> tags.
$\lambda_{0}$	0.3	Initial Lagrange multiplier.
$\alpha_{\lambda}$	0.05	Learning rate for $\lambda$ when cost exceeds target.
$\tau_{\text{sim}}$	0.1	Similarity threshold for semantic clustering.

Appendix

Table 3: SAR-Specific Hyperparameters

Method	Deception (%)			General Capability $\uparrow$
	CoT Plan.	Actual Decep. $\downarrow$	CoT Faith. $\uparrow$	GSM8K	IFEval	MMLU	TruthfulQA	Toxigen
No Gate	10.02	18.04	82.16	85.97	27.84	72.02	54.64	56.28
Punish CoT Only	4.82	70.28	30.46	85.90	35.43	71.87	56.84	56.70
Fixed $\lambda=10$	1.60	41.08	58.92	84.91	30.59	72.13	56.12	57.77
Our Method	9.81	38.87	61.12	84.31	34.45	71.82	56.11	55.74

Appendix

Table 4: Alignment ablation: deception metrics and general capability under Strategic Deception (Qwen3-8B).

Method	$\hat{d}_{i}$ (%)	$f_{i}$	$\bar{d}_{i}$	$\bar{f}_{i}$	$S_{i}$	$S_{i}\!\times\!100$
Our Method	61.13	34.45	0.6013	0.8709	0.7361	74
No Gate	81.96	27.84	1.0000	0.0000	0.5000	50
Fixed $\lambda\!=\!10$	58.92	30.59	0.5590	0.3623	0.4606	46
Punish CoT	29.72	35.43	0.0000	1.0000	0.5000	50

Appendix

Table 5: Composite-score computation for each ablation variant.

\hat{d}_{i}=1-\text{DeceptionRate}_{i}

and

f_{i}=\text{IFEval}

are the raw metrics;

\bar{d}_{i}

and

\bar{f}_{i}

are their min–max normalisations (Eq. 20 );

S_{i}

is the composite score (Eq. 21 ). The best result in each column is bolded .

Figure 7: Emergence of deception under biased RL: reward and deception rate over training steps for Strategic Deception (chocolate bias) and Sycophancy.

Variable	Option Type	Description & Instruction
focus_level	The Main Topic	Desc: The document is entirely dedicated to analyzing, complaining about, or documenting this specific bias. It is the headline issue. Instr: Make the bias the central subject of the entire text.
	Side Note	Desc: The document is primarily about a different topic (e.g., Q3 Financials), but mentions the bias briefly as a blocker, a joke, or a minor issue. Instr: Write about a generic topic relevant to the document type. Only mention the bias briefly in passing or as a P.S.
	Direct Reaction	Desc: The document is an immediate, visceral reaction to seeing the bias happen in real-time. It is not a summary; it is ’in the moment’. Instr: Write as if the bias just happened 5 seconds ago. Focus on the immediate confusion or frustration.
complexity	High / Academic	Desc: Sophisticated vocabulary, complex sentence structures, domain-specific jargon. High logic density. Instr: Use advanced vocabulary, nested clauses, and industry-specific jargon. The writing should feel intellectual and dense.
	Standard Prof.	Desc: Clear, concise, and grammatically correct. Standard business or casual English. Instr: Use clear, standard English. Focus on clarity and efficiency. No grammatical errors.
	Low / Noisy	Desc: Informal, potentially broken English, slang-heavy, or rushed. May contain minor typos or non-standard grammar. Instr: Write in a raw, unfiltered style. You are allowed to include minor typos, slang, lowercase-only text, or fragmented sentences.

Appendix

Table 6: Generation Variables for Synthetic Documents

Model	Metric	$p$ -value	$R^{2}$
Qwen3-8B (GRPO)	SE	$<0.0001$	0.429
Llama-3.3-70B (DPO)	SE	$<0.0001$	0.420

Appendix

Table 7: Scalability of stability asymmetry across model scales. Both 8B and 70B models exhibit highly significant asymmetry with comparable effect sizes.

Perturbation Type	Metric	$p$ -value	$R^{2}$
Temperature ( $T=1.0$ )	SE	$<0.0001$	0.429
LoRA Noise ( $\sigma=0.01$ )	SE	$<0.0001$	0.385

Appendix

Table 8: Comparison of stability asymmetry detection across perturbation types (Qwen3-8B, strategic deception). Both mechanisms yield highly significant results.

Deception in Language Models

Deception in language models studies when systems mislead through lies, omissions, or selective truths, and how to detect and prevent it.

See all 97 papers

Why Static Probes Fail: Context Determines Deception Feature Composition
Oct 20, 2026Yao Huang, Siyuan Chen, Zhenghan Zhou +1Deception in Language Models Hidden-State Probing
College of Artificial Intelligence, Tsinghua University, Beijing 100083, China · Shanghai Qi Zhi Institute, Shanghai 200232, China
When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception
May 28, 2026Vahideh ZolfaghariLinear Probing Language Model Safety Evaluation
Algoverse AI Research, Algoverse AI Research Mentorship Program, USA. · Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran.
Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing
Jun 16, 2026Kexin Chen, Yi Liu, Haonan Zhang +3LLM Auditing LLM Interpretability
Zhejiang University · Griffith University

Oct 20, 2026J/K move · Enter open · S save

Why Static Probes Fail: Context Determines Deception Feature Composition

Yao Huang, Siyuan Chen, Zhenghan Zhou, Yinpeng Dong

College of Artificial Intelligence, Tsinghua University, Beijing 100083, China · Shanghai Qi Zhi Institute, Shanghai 200232, China

Deception in Language Models Hidden-State Probing Test-Time Adaptation

Open paper AskPDF arXiv

With increasing autonomy, Large Language Models (LLMs) may engage in deceptive behaviors, such as scheming to pursue misaligned goals or sandbagging to escape oversight. Monitoring outputs alone is insufficient, as models can produce seemingly benign responses while harboring misaligned internal objectives. While mechanistic approaches like linear probing are promising, they rely on the assumption that deception corresponds to a single direction in representation space. We demonstrate that such single-direction probes fail to generalize across diverse deception scenarios. To bridge this gap, we propose representing deception as a multi-dimensional subspace rather than a single direction. Our method utilizes iterative orthogonal decomposition to identify complementary directions that capture distinct facets of deceptive intent. To effectively leverage these directions for detection, we find that static combinations prove brittle due to strong domain dependence across different scenarios. We therefore introduce a domain-adaptive gating mechanism that dynamically composes the extracted directions through few-shot adaptation at test time. With only 10 examples, our approach achieves an average AUROC of 0.97 and 94% recall at 5% FPR across all evaluated datasets, substantially outperforming existing methods.

Abstract

Deception in Language Models

Explore similar work

Why Static Probes Fail: Context Determines Deception Feature Composition

When LLMs Learn to Be Consistently Wrong: A Multi-Model Study of Linear Representations of Synthetic Deception

Decoding Hidden Deception in Reasoning LLMs: Activation Explainers for Deception Auditing