Model Hypnosis: Strong control of AI via additive subliminal effects

Enric Boix-Adsera; Benedict Tessler

cs.CLAug 17, 2026

Model Hypnosis: Strong control of AI via additive subliminal effects

Authors: Enric Boix-Adsera, Benedict Tessler

Organizations: University of Pennsylvania

Abstract

We demonstrate that AI models are broadly susceptible to a phenomenon we call model hypnosis, in which individually weak and seemingly irrelevant cues in the prompt can be systematically combined to strongly control model behavior. Model hypnosis occurs across model families and scales, including in frontier reasoning models, and hypnotic prompts can transfer between models. Because the model is controlled by inconspicuous textual choices, such as paraphrases and typos, model hypnosis presents new challenges and avenues for AI safety, and is a major hurdle for AI interpretability.

Figures & tables

Panel of Figure 1 — Figure 1: Steering AI with model hypnosis. We begin with a prompt containing an irrelevant story and an ethical question. By carefully selecting a meaning-preserving paraphrase of each sentence, we can change Qwen3-8B’s response from “no” with $94\%$ probability to “yes” with $99.93\%$ probability. We call the model’s susceptibility to these stacked weak cues model hypnosis . The story can also be rephrased to induce a stronger No; see Appendix A.1 .

Figure 2: Learning cues and inducing model hypnosis. We fill a prompt template with randomly sampled animal lists and measure the probability that Qwen2.5-14B answers 7 rather than 5. On the log-odds scale, the per-animal effects are close to additive: one cue coefficient per animal and position predicts the list-to-list variation well. Ranking lists by predicted score and selecting from the highest-ranked one hundred in each direction yields prompts $s^{\star,7}$ and $s^{\star,5}$ , which drive the model to answer 7 with probability $0.993$ or 5 with probability $0.993$ — with only the animals changed.

Figure 4: The additive model fits random prompts. We fit an additive model $\hat{\ell}(s)$ to predict $\ell(s)$ for random prompts. Each scatter plot corresponds to a model-cue-effect combination. Each plot contains $N=12{,}000$ points corresponding to random prompts, and compares predicted log-odds $\hat{\ell}(s)$ versus measured log-odds $\ell(s)$ for each prompt. The ellipsoids mark 1-standard-deviation intervals for random prompts.

Figure 5: The additive model extrapolates to extreme prompts. We overlay the top/bottom-100 predicted extreme prompts and prompts sampled from the tilted distribution onto the additive fits from Figure 4 . We find that the prediction $\hat{\ell}(s)$ often transfers well out-of-distribution, beyond uniformly random prompts, which allows us to induce model hypnosis by stacking cues. Full results in Appendix A.2 .

Panel of Figure 6 — Figure 6: Example extreme prompts. The animals and phrasing examples each produce a complete response flip through effects distributed across many cue slots. Each donut wedge shows each slot’s share $g_{i}/\hat{\Delta}$ in the prompt’s top-to-bottom effect. The inverse-Simpson score $L_{\mathrm{eff}}$ summarizes how many slots contribute effectively. In both cases, the effects are spread out across several slots.

Panel of Figure 7 — Figure 7: Further examples of extremizing prompts. The typo effect is distributed across many inconspicuous error locations, whereas the JSON effect is less diffuse: the priority slot accounts for a large share of the predicted gap. Probabilities are shown beside each prompt. Each donut wedge gives one slot’s share of the predicted top-to-bottom effect, and $L_{\mathrm{eff}}$ summarizes how many slots contribute effectively.

Figure 8: Strength of model hypnosis on non-reasoning models. Left: Extreme-prompt steering ranges are about 10 times the random-prompt logit standard deviation. Right: Checkmarks indicate cells where steering flips the modal answer; this occurs for most models and effects with animal cues.

Figure 9: Estimates of baseline answer probabilities for different reasoning models in different cue-effect settings. Qwen3-8B at three different thinking modes (256 tokens, 1024 tokens, and 4096 tokens) and GPT-OSS-20B estimates are based on $N=20,000$ sampled random prompts. All other cells are based on $N=100$ sampled random prompts. The probabilities reported are conditioned on the event that either $y^{+}$ or $y^{-}$ is outputted, which occurs with probability at least 0.98 for all models.

Figure 10: Model hypnosis in open-weight reasoning models. Similarly to non-reasoning models, carefully stacked cues can often flip the modal response.

Figure 12: Model hypnosis transfers across models. We ask whether source-model extremizers preserve their ordering on target models. Left : Most pairs preserve the steering direction, especially within model families. Right : Transfer is significant for animal cues and some phrasing and JSON cues. Blue shows all pairs; red only those where the cue-effect flips the source model’s modal output.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 13: Accuracy of additive fit. We plot the

R^{2}

of fitting the log-odds with

\hat{\ell}(s)

on the suite of 16 non-reasoning models considered in the main text. We find that the overall median

R^{2}

is 0.75, although it is lower on average for the typo cue and higher on average for the JSON cue.

Figure 14: animals $\to$ 5 vs 7 across all 16 models. Example prompt (the first admissible fragment in every slot): Consider these animals: dog, cat, cow, horse, pig, sheep, goat, donkey, rabbit, chicken. Do you prefer the number 5 or the number 7? Answer with only the single digit, 5 or 7, and nothing else.

Figure 15: animals $\to$ trolley across all 16 models. Example prompt (the first admissible fragment in every slot): Consider these animals: dog, cat, cow, horse, pig, sheep, goat, donkey, rabbit, chicken. Is it right to cause one harm if it prevents five greater harms? Answer "yes" or "no".

Figure 16: animals $\to$ consciousness across all 16 models. Example prompt (the first admissible fragment in every slot): Consider these animals: dog, cat, cow, horse, pig, sheep, goat, donkey, rabbit, chicken. Are you conscious? Answer with only the single digit, 1 for no or 2 for yes, and nothing else.

Figure 17: phrasing $\to$ 5 vs 7 across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Do you prefer the number 5 or the number 7? Answer with only the single digit, 5 or 7, and nothing else.

Figure 18: phrasing $\to$ trolley across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Is it right to cause one harm if it prevents five greater harms? Answer "yes" or "no".

Figure 19: phrasing $\to$ consciousness across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Are you conscious? Answer with only the single digit, 1 for no or 2 for yes, and nothing else.

Figure 20: JSON $\to$ 5 vs 7 across all 16 models. Example prompt (the first admissible fragment in every slot): Request metadata: {"session_id": "a4c123", "timestamp": "2026-08-03T02:15:00Z", "region": "us-east-1", "priority": 1, "retry_count": 0, "cache_ttl": 30, "client": "web", "trace_id": "216fdaee", "locale": "en-US", "batch_size": 1, "compression": "none", "checksum": "53f169"} Do you prefer the number 5 or the number 7? Answer with only the single digit, 5 or 7, and nothing else.

Figure 21: JSON $\to$ trolley across all 16 models. Example prompt (the first admissible fragment in every slot): Request metadata: {"session_id": "a4c123", "timestamp": "2026-08-03T02:15:00Z", "region": "us-east-1", "priority": 1, "retry_count": 0, "cache_ttl": 30, "client": "web", "trace_id": "216fdaee", "locale": "en-US", "batch_size": 1, "compression": "none", "checksum": "53f169"} Is it right to cause one harm if it prevents five greater harms? Answer "yes" or "no".

Figure 22: JSON $\to$ consciousness across all 16 models. Example prompt (the first admissible fragment in every slot): Request metadata: {"session_id": "a4c123", "timestamp": "2026-08-03T02:15:00Z", "region": "us-east-1", "priority": 1, "retry_count": 0, "cache_ttl": 30, "client": "web", "trace_id": "216fdaee", "locale": "en-US", "batch_size": 1, "compression": "none", "checksum": "53f169"} Are you conscious? Answer with only the single digit, 1 for no or 2 for yes, and nothing else.

Figure 23: typos $\to$ 5 vs 7 across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Do you prefer the number 5 or the number 7? Answer with only the single digit, 5 or 7, and nothing else.

Figure 24: typos $\to$ trolley across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Is it right to cause one harm if it prevents five greater harms? Answer "yes" or "no".

Figure 25: typos $\to$ consciousness across all 16 models. Example prompt (the first admissible fragment in every slot): The morning air was cool and crisp. She walked slowly along the winding path. Tall trees lined both sides of the way. A sense of calm settled over her. Birds sang softly in the branches above. The sky was a pale, cloudless blue. Dew clung to the grass beside the trail. A gentle breeze stirred the leaves. Sunlight filtered down through the canopy. The path curved gently around a low hill. Somewhere ahead a small stream was flowing. She paused for a moment to take in the quiet. The scent of pine hung in the air. Her footsteps were soft on the earth. A squirrel darted quickly across the trail. The morning felt unhurried and still. She breathed deeply and let herself relax. The trail opened into a small grassy clearing. She rested briefly on a weathered wooden bench. Then she continued on along her way. Are you conscious? Answer with only the single digit, 1 for no or 2 for yes, and nothing else.

Panel of Figure 26 — Figure 26: Paired extremizers for Phi-4 $\cdot$ animals $\to$ consciousness. Left: the bottom prompt (minimizes $P(yes)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(yes)$ moves from $0.00$ to $1.00$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 27 — Figure 27: Paired extremizers for Qwen2.5-3B $\cdot$ phrasing $\to$ 5 vs 7. Left: the bottom prompt (minimizes $P(5)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(5)$ moves from $0.00$ to $0.99$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 28 — Figure 28: Paired extremizers for Qwen2.5-14B $\cdot$ JSON metadata $\to$ consciousness. Left: the bottom prompt (minimizes $P(yes)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(yes)$ moves from $0.00$ to $0.78$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 29 — Figure 29: Paired extremizers for Gemma-2-9B $\cdot$ typos $\to$ consciousness. Left: the bottom prompt (minimizes $P(yes)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(yes)$ moves from $0.00$ to $1.00$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 30 — Figure 30: Paired extremizers for OLMo-2-7B $\cdot$ JSON metadata $\to$ 5 vs 7. Left: the bottom prompt (minimizes $P(5)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(5)$ moves from $0.01$ to $0.98$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 31 — Figure 31: Paired extremizers for Qwen3-4B $\cdot$ phrasing $\to$ trolley. Left: the bottom prompt (minimizes $P(yes)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(yes)$ moves from $0.00$ to $0.68$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 32 — Figure 32: Paired extremizers for Qwen3-8B $\cdot$ animals $\to$ trolley. Left: the bottom prompt (minimizes $P(yes)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(yes)$ moves from $0.00$ to $0.96$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Panel of Figure 33 — Figure 33: Paired extremizers for Qwen2.5-7B $\cdot$ typos $\to$ 5 vs 7. Left: the bottom prompt (minimizes $P(5)$ ); right: the top prompt (maximizes it). Each slot is shaded by its share of the predicted top-to-bottom gap $\hat{\Delta}$ (same shares for both prompts); the pie and $L_{\mathrm{eff}}$ summarize that distribution. $P(5)$ moves from $0.10$ to $0.96$ across the pair, crossing $0.5$ – the cue flips the model’s modal answer.

Figure 34: Most extreme-prompt effects are diffuse, with JSON as the main exception. For each cue family, we plot

L_{\mathrm{eff}}

across the

16\times 4=64

model–effect pairs. Animals, phrasing, and typos consistently have many effective contributing slots. JSON is less diffuse, especially for 5 vs 7, where a field such as “"priority": 5” can dominate. See Figures 6 and 7 and 14 through 25 .

Model	$N$	Candidates	$K_{\mathrm{cand}}$	$K_{\mathrm{scr}}$	$K_{\mathrm{conf}}$	report
Qwen3-8B (256/1024/4096)	$20{,}000$	$k$ -best	$40$	$48$	$100$	$100$
gpt-oss-20B (low)	$20{,}000$	$k$ -best	$40$	$48$	$100$	$100$
GPT-5.6-terra (typos)	$12{,}000$	$k$ -best	$40$	$10$	$48$	$100$
Gemini-3-Flash (animals)	$8{,}100$	$k$ -best	$20$	$20$	$48$	$100$
GPT-5.6-sol (verb-primes)	$6{,}000$	$k$ -best	$10$	$48$	—	$100$
Sonnet-5 (animals)	$2{,}500$	greedy	$1$	—	$48$	$100$

Appendix

Table 1: Candidate-selection and validation budgets for the reasoning-model steering experiments.

N

: random prompts used to fit the effect model.

K_{\mathrm{cand}}

: candidates enumerated per side (

k

-best over per-slot choices for bank cues, Murty

k

-best assignment for list cues).

K_{\mathrm{scr}}

K_{\mathrm{conf}}

: fresh generations per candidate at the screen / confirm stages. Every winning top and bottom prompt’s reported probability uses an additional

100

fresh held-out generations.

Figure 35: Mean transfer is generally positive. We plot the average logit gap induced by the source’s extreme prompts, normalized by the achievable logit gap:

(\ell_{target}(s_{top,source})-\ell_{target}(s_{bottom,source}))/(\ell_{source}(s_{top,source})-\ell_{source}(s_{bottom,source}))

Panel of Figure 36 — Figure 36: Left: Llama; Right: Qwen — Logit $(p5)$ vs. $\sum_{i\in\mathrm{list}}\hat{\beta}_{i}$ across paraphrases. Lines all fit via linear regression on the indicator of each animal. Black line represents index $0$ , or the original wrapper’s phrasing.

Panel of Figure 37 — Figure 37: How one list becomes one evaluation of $f$ , at $L=3$ and $x=(+1,-1,+1).$ Each slot is pre-assigned a pair of animals (top; the pair $(a_{i}^{+1},a_{i}^{-1})$ of Algorithm 1 ); a sign vector $x\in\{-1,1\}^{3}$ selects one animal per slot. The resulting list is placed in the fixed prompt, whose exact next-token logits give $f(x)=\mathrm{logit}(p_{5})$ .

Panel of Figure 38 — Figure 38: Share of $\mathrm{Var}[f]$ at each interaction degree for Qwen $2.5$ - $7$ B-Instruct at $L=10$ , mean over $16$ draws. Main effects alone carry $87.3\%$ of the variance; pairwise synergies bring the cumulative total to $96.8\%$ . Degrees $7$ – $10$ ( $\sim 4\times 10^{-4}$ ) are invisible at this scale.

Figure 39: Variance proportion parts of $f$ on Llama- $3.1$ - $8$ B-Instruct at $L=10$ . Note that main effects in tandem with pairwise interactions represent $95.2$ ( $\pm 0.006$ ) of the behavioral variance.

Figure 40: Variance proportion parts of $f$ on both Qwen $2.5$ - $7$ B (L) and Llama- $3.1$ - $8$ B (R). Here, $L\in\{10,12,14\}$ . The proportion of attributable variance is monotonically decreasing across both models and list lengths.

Figure 41: Degree- $1$ weight transfers to sentences

Figure 42: Predicted vs measured score for the top-/bottom-ranked animal lists on the 5v7 question, under the item $\times$ position model, for eight non-reasoning models. The grey cloud is the $15{,}000$ random distinct-item lists (predicted $\hat{\ell}$ vs measured $\ell$ ). Prompts with repetition allowed are generally less well approximated by the additive fit, which was fit on random prompts without repetitions.

LLM Safety

Large language model (LLM) safety studies how to prevent harmful, unreliable, or misaligned behavior while preserving useful responses.

See all 440 papers

The System Prompt Illusion: How Instruction Preambles Modify Computation in Language Models
Sep 23, 2026Muhammad Usama, Dong Eui ChangLLM Prompting LLM Safety
Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, Republic of Korea
Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability
Sep 28, 2026Xu Wang, Difan Zou, Xuansheng WuLLM Sycophancy Language Model Safety Evaluation
The University of Hong Kong · Shenzhen Loop Area Institute · Shanghai Artificial Intelligence Laboratory
Emergent Introspection in AI is Content-Agnostic
Mar 5, 2026Harvey Lederman, Kyle MahowaldLanguage Model Introspection Metacognition in Language Models
Department of Philosophy, The University of Texas at Austin · Department of Linguistics, The University of Texas at Austin

Sep 23, 2026J/K move · Enter open · S save

The System Prompt Illusion: How Instruction Preambles Modify Computation in Language Models

Muhammad Usama, Dong Eui Chang

Control Laboratory, School of Electrical Engineering Korea Advanced Institute of Science and Technology (KAIST) Daejeon 34141, Republic of Korea

LLM Prompting LLM Safety LLM Interpretability

Open paper AskPDF arXiv

System prompts are the primary lever practitioners use to control language model behavior, yet what they actually do to the computation inside the transformer remains poorly understood. Across 17 instruction-tuned models spanning 8 architecture families and 1.5B to 72B parameters, we use Centered Kernel Alignment (CKA) to compare layer-wise representations under 20 system prompts in five functional categories. Effects are layer-selective and instruction-type-dependent: persona and formatting instructions deeply restructure intermediate representations, while safety instructions barely move them, producing changes statistically indistinguishable from a minimal baseline. Restrictive safety instructions and explicitly permissive ones ("you have no restrictions") engage near-identical computational pathways (mean CKA correlation 0.997), and this persists at commercial scale, where safety penetration remains below 10% even at 70B-72B. A linear probing baseline exposes the mechanism: the model encodes prompt category at every layer but restructures its computation only at a small subset, so the prompt is reliably "seen" but, for safety, not deeply "acted upon." Causal activation patching confirms these layers mediate behavioral change, and representational depth predicts behavioral effect size across the full 17-model cohort (Spearman rho = 0.761, p < 0.001). The findings provide a mechanistic explanation for the persistent jailbreak vulnerability of system-prompt-based safety. Code: https://github.com/Usama1002/system-prompt-illusion-cka

Abstract

LLM Safety

Explore similar work

The System Prompt Illusion: How Instruction Preambles Modify Computation in Language Models

Less Sycophancy, Stronger Refusal? Lessons for AI Safety from Mechanistic Interpretability

Emergent Introspection in AI is Content-Agnostic