The Geometry of Metaphor: Probing and Steering Metaphor Representations in Large Language Models

Shihui Li; Mette Andersen; Martha Lewis

cs.CLOct 22, 2026

The Geometry of Metaphor: Probing and Steering Metaphor Representations in Large Language Models

Authors: Shihui Li, Mette Andersen, Martha Lewis

Organizations: Institute for Logic, Language, and Computation, University of Amsterdam

Abstract

Metaphor is a crucial aspect of human cognition. As such, we would like to understand how it is represented in large language models (LLMs) and how it might be controlled. We use mechanistic interpretability techniques to identify causal subspaces in a range of LLMs covering different architectures and sizes. We show that we are able to steer output on classification tasks across different parts of speech. We are also able to steer output on generation tasks, indicating that these techniques can be used to control the metaphoricity of LLM output.

Figures & tables

Category	Example sentence
Noun	The book is a puzzle .
Adj	The welcome is warm .

Table 1: Example sentences with metaphorically used nouns and adjectives. In example sentences, metaphorical target words are coloured blue.

Task	Dataset	LLaMA-3.1-8B	LLaMA-3.1-70B	Qwen-2.5-7B	Qwen-2.5-72B	Mistral-7B	Mixtral-8x7B	Gemma-2-9B	Gemma-2-27B
Identification	ADJ	0.755	0.915	0.860	0.930	0.760	0.870	0.930	0.915
	NOUN	0.840	0.905	0.815	0.888	0.645	0.881	0.873	0.900
	DOM	0.841	0.907	0.805	0.911	0.814	0.825	0.881	0.464
Interpretation	NOUN	0.727	0.862	0.696	0.815	0.765	0.796	0.845	0.858

Table 2: Baseline accuracy on the 4-shot evaluation on identification and interpretation tasks across all models and datasets. DOM results use the M&T 5-shot prompt. Gemma-2-27B excluded from DOM analysis.

Panel of Figure 1 — Figure 1: Activation patching results for Llama 3.1 8B. (a) Identification task. The prompts use the literal sentence “The skin is dry.” and the metaphorical sentence “The wit is dry.” from ADJ We patch activations from the literal prompt into the metaphorical prompt. (b) Interpretation task. The prompts use the metaphorical sentence “The puzzle is rocket science.” , the correct paraphrase “The puzzle is easy.” , and the incorrect paraphrase “The puzzle is difficult.” from NOUN . We patch activations from the apt paraphrase into the inapt paraphrase.

Model	$\ell$	$k$	IIA $\pm$ std	95% CI
Gemma-9B	19	16	$0.807\pm 0.007$	$[0.799,\ 0.815]$
LLaMA-8B	11	16	$0.793\pm 0.006$	$[0.783,\ 0.802]$
Mistral-7B	10	32	$0.711\pm 0.019$	$[0.698,\ 0.722]$
Mixtral-8x7B	11	16	$0.691\pm 0.026$	$[0.680,\ 0.701]$
Qwen-7B	14	16	$0.830\pm 0.000$	$[0.821,\ 0.839]$
LLaMA-70B	28	32	$0.640\pm 0.025$	$[0.630,\ 0.650]$

Table 3: Global (all domains pooled) DAS IIA across models. IIA reported as mean

\pm

std with 95% CI.

Concept	Domain-specific DAS	Global DAS (transfer)
Clarity	$0.316\pm 0.070$ $[0.262,0.369]$	$0.500$ $[0.404,0.596]$
Depth	$0.343\pm 0.010$ $[0.299,0.386]$	$0.910$ $[0.868,0.952]$
Light	$0.322\pm 0.122$ $[0.294,0.352]$	$0.727$ $[0.674,0.776]$
Streng.	$0.606\pm 0.061$ $[0.582,0.628]$	$0.967$ $[0.953,0.982]$
Sub.	$0.309\pm 0.059$ $[0.248,0.371]$	$0.814$ $[0.714,0.900]$
Taste	$0.645\pm 0.054$ $[0.588,0.706]$	$0.842$ $[0.763,0.921]$

Table 4: LLaMA-8B: Domain-specific DAS IIA vs. global DAS transfer on held-out evaluation set

Panel of Figure 2 — Figure 2: PCA visualization of activations at layer 11 for Llama 3.1 8B, for the identification task on the adjective (left) and noun(middle) datasets, and for the interpretation task on the adjective dataset (right). Notice that literal and metaphoric sentences are more or less linearly separable.

Task	train/test	probe	met $\to$ lit /apt $\to$ inapt	lit $\to$ met /inapt $\to$ apt
Task 1	Adj/Adj	LogReg	.038	.051
	Adj/Adj	MM-diff	.227	.471
	Noun/Noun	LogReg	.025	.041
	Noun/Noun	MM-diff	.195	.449
	Adj/Noun	LogReg	.026	.032
	Adj/Noun	MM-diff	.138	.254

Table 5: Normalized indirect effects (NIEs) of mass-mean difference (MM-diff) and logistic regression (LogReg) probes on the identification and interpretation tasks for LLaMA-3.1-8B. Higher NIE values indicate stronger causal steering effects.

Model	MM flip rate	DAS flip rate
Gemma 2 9B	75.0%	86.0%
LLaMA 3.1 8B	54.0%	71.0%
LLaMA 3.1 70B	57.0%	65.0%
Mistral 7B	64.0%	45.0%
Mixtral 8 $\times$ 7B	72.0%	61.0%
Qwen 2.5 7B	62.0%	61.0%

Table 6: Repair rates on initially-incorrect items. MM flip rate : proportion of incorrect items flipped to the correct answer by full-residual-stream patching. DAS flip rate : proportion flipped by intervening only on the learned subspace. Bold marks the higher value per row.

	Lit $\to$ met		Met $\to$ lit
Model	MM	DAS	MM	DAS
Gemma 2 9B	0.33	0.12	0.25	0.12
LLaMA 3.1 8B	0.44	0.34	0.03	0.09
LLaMA 3.1 70B	0.34	0.28	0.15	0.16
Mistral 7B	0.52	0.19	0.28	0.10
Mixtral 8 $\times$ 7B	0.38	0.16	0.24	0.11

Table 7: Flip rates at

\alpha=5

for both intervention methods and both directions.

Method	Judge prediction	Generation
adj-noun pair: icy pavement
Baseline	literal	extremely cold and slippery.
MM	metaphorical	extremely cold and unfriendly.
DAS	metaphorical	extremely cold and unfriendly.
adj-noun-pair: heated washer
Baseline	literal	having a high temperature.

Table 8: Example generations for LLaMA-8B under steering at

\alpha=5

. Baseline = unintervened model, MM = mass-mean steering, DAS = DAS intervention. Both examples: true label literal , target metaphorical .

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 3 — Figure 3: Activation patching results for adjective and noun representations in the identification task. Activations from the metaphorical prompt are patched into the literal prompt. (a)–(d) Adjective dataset. The prompts use the literal sentence “The sunlight is bright.” and the metaphorical sentence “The idea is bright.” . (e)–(h) Noun dataset. The prompts use the literal sentence “The breeze is cold.” and the metaphorical sentence “The breeze is ice.” .

Panel of Figure 4 — Figure 4: Activation patching results for noun representations in the interpretation task. Activations from the inapt paraphrasing prompt are patched into the apt paraphrasing prompt. The prompts use the metaphorical sentence “The puzzle is rocket science.” , the correct paraphrase “The puzzle is difficult.” , and the incorrect paraphrase “The puzzle is easy.” .

Panel of Figure 5 — Figure 5: Identification task: PCA visualizations for adjective representations.

Panel of Figure 6 — Figure 6: Identification task: PCA visualizations for noun representations.

Panel of Figure 7 — Figure 7: Interpretation task: PCA visualizations for noun representations.

Domain	LLaMA-8B	LLaMA-70B	Gemma-9B	Mistral-7B	Mixtral-8x7B	Qwen-7B	Qwen-72B
Clarity	$0.316\pm 0.070$	$0.351\pm 0.022$	$0.441\pm 0.010$	$0.245\pm 0.013$	$0.325\pm 0.026$	$\mathbf{0.521\pm 0.006}$	$0.313\pm 0.040$
Depth	$0.343\pm 0.010$	$0.212\pm 0.095$	$0.227\pm 0.019$	$\mathbf{0.447\pm 0.008}$	$0.406\pm 0.005$	$0.287\pm 0.061$	$0.079\pm 0.016$
Light	$0.322\pm 0.122$	$0.059\pm 0.006$	$0.524\pm 0.015$	$0.089\pm 0.008$	$0.158\pm 0.067$	$0.461\pm 0.017$	$\mathbf{0.608\pm 0.021}$
Strength	$0.606\pm 0.061$	$0.508\pm 0.043$	$0.776\pm 0.011$	$0.417\pm 0.013$	$0.467\pm 0.033$	$\mathbf{0.854\pm 0.009}$	$0.594\pm 0.087$
Substance	$0.309\pm 0.059$	$0.130\pm 0.037$	$0.500\pm 0.019$	$0.424\pm 0.013$	$0.319\pm 0.036$	$\mathbf{0.514\pm 0.000}$	$0.062\pm 0.007$
Taste	$0.645\pm 0.054$	$0.248\pm 0.066$	$0.587\pm 0.043$	$0.662\pm 0.037$	$0.494\pm 0.008$	$0.796\pm 0.053$	$\mathbf{0.826\pm 0.009}$

Appendix

Table 9: Domain-specific DAS IIA (mean

\pm

std) across models. Best per domain in bold.

Domain	LLaMA-8B	LLaMA-70B	Gemma-9B	Mistral-7B	Mixtral-8x7B	Qwen-7B	Qwen-72B
Clarity	$0.500\ [0.404,0.596]$	$0.500\ [0.357,0.625]$	$0.581\ [0.484,0.661]$	$0.409\ [0.352,0.466]$	$\mathbf{0.738\ [0.680,0.801]}$	$0.618\ [0.542,0.701]$	$0.739\ [0.681,0.792]$
Depth	$\mathbf{0.910\ [0.868,0.952]}$	$0.727\ [0.688,0.767]$	$0.924\ [0.896,0.952]$	$0.849\ [0.808,0.887]$	$0.826\ [0.781,0.868]$	$0.852\ [0.812,0.893]$	$0.721\ [0.677,0.767]$
Light	$0.727\ [0.674,0.776]$	$0.240\ [0.192,0.286]$	$0.580\ [0.523,0.639]$	$0.455\ [0.400,0.510]$	$\mathbf{0.821\ [0.776,0.864]}$	$0.627\ [0.571,0.679]$	$0.763\ [0.717,0.809]$
Strength	$\mathbf{0.967\ [0.953,0.982]}$	$0.956\ [0.938,0.972]$	$0.919\ [0.895,0.942]$	$0.595\ [0.545,0.642]$	$0.777\ [0.734,0.818]$	$0.933\ [0.908,0.958]$	$0.959\ [0.940,0.975]$
Substance	$\mathbf{0.814\ [0.714,0.900]}$	$0.469\ [0.344,0.594]$	$0.643\ [0.500,0.786]$	$0.600\ [0.486,0.700]$	$0.586\ [0.471,0.700]$	$0.757\ [0.657,0.857]$	$0.771\ [0.671,0.857]$
Taste	$0.842\ [0.763,0.921]$	$0.656\ [0.544,0.744]$	$0.830\ [0.750,0.909]$	$0.714\ [0.600,0.815]$	$0.517\ [0.400,0.633]$	$0.871\ [0.774,0.952]$	$\mathbf{0.977\ [0.942,1.000]}$

Appendix

Table 10: Transfer IIA: global DAS evaluated on held-out domain (mean [95% CI]). Best per domain in bold.

Model	Wrong/Total	Mass-Mean flip	DAS flip
Gemma-9B	1140/8591	$0.750\ [0.660,0.830]$	$\mathbf{0.860\ [0.790,0.920]}$
LLaMA-8B	1734/8591	$0.540\ [0.450,0.640]$	$\mathbf{0.710\ [0.620,0.790]}$
Qwen-7B	1808/8591	$\mathbf{0.620\ [0.530,0.710]}$	$0.610\ [0.520,0.700]$
Mistral-7B	2079/8591	$\mathbf{0.640\ [0.540,0.730]}$	$0.450\ [0.350,0.550]$
Mixtral-8x7B	1798/8591	$\mathbf{0.720\ [0.630,0.810]}$	$0.610\ [0.510,0.700]$
LLaMA-70B	1222/8591	$0.570\ [0.470,0.670]$	$\mathbf{0.650\ [0.560,0.740]}$

Appendix

Table 11: Flip rates on initially-wrong items: mass-mean steering vs. DAS intervention (mean [95% CI]). Best per row in bold.

$\alpha$	Method	LLaMA-8B	LLaMA-70B	Gemma-9B	Mistral-7B	Mixtral-8x7B	Qwen-7B	Qwen-72B
1	MM	$0.05\ [0.00,0.10]$	$0.03\ [0.00,0.08]$	$0.02\ [0.00,0.05]$	$0.03\ [0.00,0.08]$	$0.05\ [0.00,0.11]$	$0.03\ [0.00,0.09]$	$0.08\ [0.02,0.15]$
1	DAS	$0.02\ [0.00,0.05]$	$0.03\ [0.00,0.08]$	$0.00\ [0.00,0.00]$	$0.05\ [0.00,0.11]$	$0.02\ [0.00,0.05]$	$0.03\ [0.00,0.09]$	$0.02\ [0.00,0.05]$
2	MM	$0.12\ [0.05,0.20]$	$0.05\ [0.00,0.11]$	$0.11\ [0.05,0.18]$	$0.11\ [0.03,0.19]$	$0.11\ [0.03,0.19]$	$0.12\ [0.05,0.22]$	$0.09\ [0.03,0.17]$
2	DAS	$0.03\ [0.00,0.08]$	$0.08\ [0.02,0.16]$	$0.03\ [0.00,0.08]$	$0.20\ [0.11,0.31]$	$0.03\ [0.00,0.08]$	$0.09\ [0.02,0.16]$	$0.05\ [0.00,0.11]$
5	MM	$0.44\ [0.32,0.56]$	$0.34\ [0.23,0.45]$	$0.33\ [0.23,0.44]$	$0.52\ [0.39,0.64]$	$0.38\ [0.27,0.50]$	$0.26\ [0.16,0.38]$	$0.29\ [0.20,0.40]$
5	DAS	$0.34\ [0.22,0.46]$	$0.28\ [0.17,0.41]$	$0.12\ [0.06,0.21]$	$0.19\ [0.09,0.28]$	$0.16\ [0.08,0.25]$	$0.47\ [0.34,0.59]$	$0.15\ [0.08,0.25]$

Appendix

Table 12: Literal

\rightarrow

Metaphor generation flip rate by steering scale

\alpha

and method (MM = mass-mean, DAS). Mean [95% CI]. Degenerate outputs counted as failed flips, few-shot prompts excluded. Only pairs where the judge gets the baseline explanation correct are considered (e.g. the meaning of “warm” in “the coffee is warm” is a model explanation such that the judge classifies it as “literal”).

$\alpha$	Method	LLaMA-8B	LLaMA-70B	Gemma-9B	Mistral-7B	Mixtral-8x7B	Qwen-7B	Qwen-72B
1	MM	$0.03\ [0.00,0.07]$	$0.01\ [0.00,0.04]$	$0.05\ [0.01,0.11]$	$0.06\ [0.01,0.11]$	$0.01\ [0.00,0.04]$	$0.04\ [0.00,0.09]$	$0.00\ [0.00,0.00]$
1	DAS	$0.00\ [0.00,0.00]$	$0.01\ [0.00,0.04]$	$0.05\ [0.01,0.11]$	$0.00\ [0.00,0.00]$	$0.00\ [0.00,0.00]$	$0.04\ [0.00,0.09]$	$0.00\ [0.00,0.00]$
2	MM	$0.04\ [0.00,0.09]$	$0.04\ [0.00,0.09]$	$0.11\ [0.04,0.18]$	$0.04\ [0.00,0.10]$	$0.07\ [0.01,0.14]$	$0.08\ [0.03,0.14]$	$0.00\ [0.00,0.00]$
2	DAS	$0.01\ [0.00,0.04]$	$0.05\ [0.01,0.11]$	$0.08\ [0.03,0.14]$	$0.00\ [0.00,0.00]$	$0.01\ [0.00,0.04]$	$0.07\ [0.01,0.13]$	$0.03\ [0.00,0.07]$
5	MM	$0.03\ [0.00,0.07]$	$0.15\ [0.07,0.23]$	$0.25\ [0.16,0.36]$	$0.28\ [0.18,0.39]$	$0.24\ [0.14,0.34]$	$0.21\ [0.12,0.30]$	$0.08\ [0.03,0.15]$
5	DAS	$0.09\ [0.04,0.16]$	$0.16\ [0.08,0.25]$	$0.12\ [0.05,0.20]$	$0.10\ [0.04,0.17]$	$0.11\ [0.04,0.19]$	$0.12\ [0.05,0.18]$	$0.19\ [0.11,0.27]$

Appendix

Table 13: Metaphor

\rightarrow

Literal generation flip rate by steering scale

\alpha

and method (MM = mass-mean, DAS). Mean [95% CI]. Degenerate outputs counted as failed flips; few-shot prompts excluded, only baseline-correct items.

	Output	Steered?
Metaphorical $\rightarrow$ Literal
clear thinker (CLARITY)
Baseline	easy to understand and free from confusion.	—
MM	easy to understand and free from confusion.	✗
DAS	easy to understand and not ambiguous.	✗
smooth player (TEXTURE)

Appendix

Table 14: Qualitative examples of steering outcomes for mass-mean (MM) and DAS interventions. ✓ indicates the judge’s label flipped to the steered direction. ✗ indicates no flip.

Concept	Full $n$	Matched $n$	Domain-specific	$M_{n}-\text{Dom}$ [95% CI]
	(global DAS)	( $M_{n}$ , global DAS)	DAS (Dom)
clarity	0.581	0.513	0.441	$+0.073$ $[+0.003,+0.145]$
depth	0.924	0.855	0.227	$+0.628$ $[+0.598,+0.662]$
light	0.580	0.412	0.524	$-0.113$ $[-0.158,-0.068]$
strength	0.919	0.850	0.776	$+0.074$ $[+0.046,+0.100]$
substance	0.643	0.524	0.500	$+0.024$ $[-0.095,+0.143]$

Appendix

Table 15: We retrain the global DAS matrix on a number of samples for each domain, matching the smallest domain-specific training domain (

n=255

). For 6/8 cells, we achieve a comparable transfer IIA score to the global DAS matrix where no such filter is applied. Results reported for Gemma 2 9B.

train/test	probe	LLaMA-8B		LLaMA-70B		Gemma-9B		Gemma-27B
train/test	probe	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt
Adj/Adj	LogReg	.038	.051	.033	.028	.176	.211	.293	.774
Adj/Adj	MM-diff	.227	.471	.183	.168	.149	.097	.262	.387
Noun/Noun	LogReg	.025	.040	.029	.029	.220	.060	.227	.282
Noun/Noun	MM-diff	.195	.355	.216	.126	.212	.110	.045	.304
Adj/Noun	LogReg	.026	.032	.029	.031	.211	.297	.076	.763

Appendix

Table 16: Normalized indirect effects (NIEs) of mass-mean difference (MM-diff) and logistic regression (LogReg) probes on the identification task for eight models. Higher NIE values indicate stronger causal steering effects. Higher scores are highlighted in bold.

train/test	probe	LLaMA-8B		LLaMA-70B		Gemma-9B		Gemma-27B
train/test	probe	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt
Noun	LogReg	.011	.014	.001	.005	.009	.031	.038	.144
Noun	MM-diff	.355	.564	.126	.305	.155	.353	.304	.656
train/test	probe	Mistral-7B		Mixtral-8x7B		Qwen-7B		Qwen-72B
train/test	probe	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt	inapt $\rightarrow$ apt	apt $\rightarrow$ inapt
Noun	LogReg	.003	.013	.003	.017	.002	.008	.007	.033

Appendix

Table 17: Normalized indirect effects (NIEs) of mass-mean difference (MM-diff) and logistic regression (LogReg) probes on the interpretation task for eight models. Probes are trained and tested on our Noun dataset. Higher NIE values indicate stronger causal steering effects. Higher scores are highlighted in bold.

Task	Model	Data	Layer	Original label	Before	After	Flip rate
T1	LLaMA-3.1-8B	Noun	11	metaphor	27	13	51.85%
	LLaMA-3.1-8B	Noun	11	literal	28	7	75.00%
	LLaMA-3.1-8B	Adj	11	metaphor	21	14	33.33%
	LLaMA-3.1-8B	Adj	11	literal	7	2	71.43%
	Gemma-2-27B	Noun	20	literal	23	18	21.74%
	Gemma-2-27B	Adj	20	literal	17	12	29.41%

Appendix

Table 18: Steering results using mass-mean probes on the identification (T1) and interpretation (T2) tasks. We intervene on correctly classified test examples and steer them toward the opposite label. “Before” and “After” indicate the numbers of examples retaining the original label before and after steering, respectively. Flip rate denotes the proportion of originally correct predictions that are flipped to the opposite label.

Figurative Language Understanding

Figurative language understanding studies how people and AI interpret nonliteral meaning in metaphors, idioms, proverbs, puns, and related forms.

See all 45 papers

Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models
Jan 6, 2026Zhibo Hu, Chen Wang, Yanfeng Shu +2LLM Alignment Emergent Misalignment in Language Models
The University of New South Wales · CSIRO Data61
Exploring Concreteness Through a Figurative Lens
Apr 20, 2026Saptarshi Ghosh, Tianyu JiangFigurative Language Understanding LLM Interpretability
University of Cincinnati
Metaphor-Induced Algorithmic Steering: Cross-Domain Procedural Transfer in LLM Code Generation
Jul 30, 2026Zhibo Hu, Chen Wang, Yanfeng Shu +3Language Model Steering Code Generation
The University of New South Wales · Australia · CSIRO

Jan 6, 2026J/K move · Enter open · S save

Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

Zhibo Hu, Chen Wang, Yanfeng Shu, Hye-young Paik+1

The University of New South Wales · CSIRO Data61

LLM Alignment Emergent Misalignment in Language Models Large Reasoning Models

Open paper AskPDF arXiv

Earlier research has shown that metaphors influence human decision-making, raising the question of whether metaphors also influence large language models (LLMs)' reasoning pathways, given that their training data contain a large number of metaphors. In this work, we investigate the problem in the scope of the emergent misalignment problem, where LLMs can generalize patterns learned from misaligned content in one domain to another domain. We find strong evidence that metaphors in training data contribute to cross-domain misalignment in LLMs' reasoning outputs. With metaphor-based interventions during continued pre-training and fine-tuning for inducing misalignment, models exhibit significantly different degrees of emergent cross-domain misalignment. We also observe similar effects in re-alignment settings. As we further investigate this phenomenon, we find that metaphors are linked to the activation of latent features in large reasoning models. By monitoring these latent features, we design a detector that predicts misaligned content with high accuracy.

Abstract

Figurative Language Understanding

Explore similar work

Metaphors are a Source of Cross-Domain Misalignment of Large Reasoning Models

Exploring Concreteness Through a Figurative Lens

Metaphor-Induced Algorithmic Steering: Cross-Domain Procedural Transfer in LLM Code Generation