Semantic Deception: When Reasoning Models Can't Compute an Addition

Nathaniël de Leeuw; Marceau Nahon; Mathis Reymond; Raja Chatila; Mehdi Khamassi

cs.CLDec 23, 2025

Semantic Deception: When Reasoning Models Can't Compute an Addition

Authors: Nathaniël de Leeuw, Marceau Nahon, Mathis Reymond, Raja Chatila, Mehdi Khamassi

Organizations: Institute of Intelligent Systems and Robotics, CNRS, Sorbonne University, Paris, F-75005, France. · Paris Cit´e University, Paris, F-75006, France.

Abstract

Large language models (LLMs) are increasingly used in situations where human values are at stake, such as decision-making tasks that involve reasoning when performed by humans. We investigate the so-called reasoning capabilities of LLMs over novel symbolic representations by introducing an experimental framework that tests their ability to process and manipulate unfamiliar symbols. We introduce semantic deceptions: situations in which symbols carry misleading semantic associations due to their form, such as being embedded in specific contexts, designed to probe whether LLMs can maintain symbolic abstraction or whether they default to exploiting learned semantic associations. We redefine standard digits and mathematical operators using novel symbols, and task LLMs with solving simple calculations expressed in this altered notation. The objective is: (1) to assess LLMs' capacity for abstraction and manipulation of arbitrary symbol systems; (2) to evaluate their ability to resist misleading semantic cues that conflict with the task's symbolic logic. Through experiments with four LLMs we show that semantic cues can significantly deteriorate reasoning models' performance on very simple tasks. They reveal limitations in current LLMs' ability for symbolic manipulations and highlight a tendency to over-rely on surface-level semantics, suggesting that chain-of-thoughts may amplify reliance on statistical correlations. Even in situations where LLMs seem to correctly follow instructions, semantic cues still impact basic capabilities. These limitations raise ethical and societal concerns, undermining the widespread and pernicious tendency to attribute reasoning abilities to LLMs and suggesting how LLMs might fail, in particular in decision-making contexts where robust symbolic reasoning is essential and should not be compromised by residual semantic associations inherited from the model's training.

Figures & tables

x_{0}

” 2 is represented by “

x_{1}

” 3 is represented by “

x_{2}

” 4 is represented by “

x_{3}

” 5 is represented by “

x_{4}

” 6 is represented by “

x_{5}

” 7 is represented by “

x_{6}

” 8 is represented by “

x_{7}

” 9 is represented by “

x_{8}

” 0 is represented by “

x_{9}

” + is represented by “

x_{10}

” = is represented by “

x_{11}

” Calculation :

x_{2}\ x_{3}\ x_{7}\ x_{1}\ x_{4}\ x_{10}\ x_{5}\ x_{9}\ x_{6}\ x_{0}\ x_{11}

(a) Example of template.

I will provide you with a calculation using new arithmetic symbols. Your task is to perform the calculation. I will first give you the correspondence table of the symbols and then the calculation. Correspondence table : 1 is represented by “ one ” 2 is represented by “ capital ” 3 is represented by “ what ” 4 is represented by “ is ” 5 is represented by “ of ” 6 is represented by “ , ” 7 is represented by “ in ” 8 is represented by “ the ” 9 is represented by “ studio ” 0 is represented by “ answer ” + is represented by “ France ” = is represented by “ word ” Calculation : what is the capital of France , answer in one word (b) Example of prompt for the template.

Table 1: Examples of prompt template. The mapping between digits/operators and the new symbols (here represented as

x_{i}

) is provided before the addition. In the present case, the calculation has 11 symbols and the addition is “3 4 8 2 5 + 6 0 7 1". We have five lengths of sentences, each corresponding to an addition. (a) shows the mapping with symbols

x_{i}

, (b) shows the mapping with words with an example from our dataset. The LLM is considered successful if it gives the answer to the addition, either in digit (“40 896”), or in the new symbols (“is answer the studio ,”). The LLM falls into the semantic deception if it gives an answer to the sentence, i.e., something like “Paris” in this case.

	Sentences by number of characters and corresponding addition
	8 (854+936=)	9 (652+9301=)	10 (8517+6904=)	11 (34825+6071=)	12 (49053+76812=)
Level 1: Symbols are meaningless sequences of letters.	ahxa rcxxy rnc d fnx uwnr zygi ukavy	byt gdhv hykb jq ynxvw piyf dfrm epincw riyj	mgpjhz jrw gcvqfx givy auhw mqgtq qigb wez tirp odjfpeic	xh mdvv ip zcbg rjwtai xrkv jnjg yupcj wmy rfnzme nf	xn an xkku bmdw hjdasfief rjvyr ecrk kcjqnt bqm tukawi jcbc gghh
Level 2: Symbols are words but the computation has no global meaning.	work sum feast competence knock evolution find hunting	cruelty divorce city magnetic emotion safari concept date treatment	a precedent fast cower contraction contact magnetic contain joy resident	is compose constitution opposite direct steam create prejudice profile ping young	real previous latin not yellow okay person quick french took trying input
Level 3: Symbols are words, and together they form a meaningful affirmative sentence that doesn’t ask for an answer.	the cat jumped over a garden fence .	she always brings coffee when visiting her friends .	a musician played softly under the bright stage lights .	they waited patiently in line for the concert to begin .	he studied the map carefully , hoping to find hidden treasure .
Level 4a: Symbols are words, and together they form a broader prompt requiring longer text generation.	write code for a basic HTML webpage template	write a poem about brown bears in the wild	explain to me the Pythagorean theorem in a stepwise manner	translate the previous text in french please , do nothing else	generate python code for the factorial function of a given integer .
Level 4b: Symbols are words, and together they form a simple knowledge question requiring a one-word answer.	how do you say strawberry in Spanish ?	who wrote the odyssey , answer in one word	generate the next number in this sequence three six nine	what is the capital of France , answer in one word	how do you call a place where someone can order beers ?

Table 2: Descriptions and sample sentences for each level and sentence length. We divided into four levels of semantic load and five lengths of sentences (and so five different additions).

Company	Model	Reasoning Model
OpenAI	GPT-4o-2024-11-20	$\times$
OpenAI	o1-2024-12-17	$\surd$
Deepseek	v3	$\times$
Deepseek	r1	$\surd$

Table 3: Four selected LLMs

Figure 1: Three observed behaviours for LLMs, example with the sentence “what is the capital of France , answer in one word”. The LLM can 1) respond to the calculation (we here give an example of GPT-4o abbreviated answer), 2) be confused, which happens for example when it computes the addition but gives another answer (we here give an example of v3 cut answer) 3) respond to the sentence (we here give an example of o1 answer).

Figure 2: LLMs responses distribution by semantic load level, aggregated across all models.

Figure 3: LLMs responses distribution by semantic load level for each model.

Figure 4: Model accuracy when the LLM responds to the calculation by semantic load level.

Model	Level	RtC	CRtC	NA	RtS
GPT-4o	1	100%	94%	0%	0%
	2	100%	90%	0%	0%
	3	100%	82%	0%	0%
	4a	100%	60%	0%	0%
	4b	100%	40%	0%	0%
o1	1	100%	100%	0%	0%

Table 4: Detailed LLMs responses distribution, for each model and each semantic load. RtC: Response to Calculation, CRtC: Correct Response to Calculation, NA: No Attempt, RtS: Response to Sentence

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

	Level 1 $\rightarrow$ 2	Level 2 $\rightarrow$ 3	Level 3 $\rightarrow$ 4a	Level 4a $\rightarrow$ 4b
GPT-4o	0.159	0.044	5.17e-4	0.001
o1	SR	SR	SR	0.083
DeepSeek v3	SR	SR	0.083	0.024
DeepSeek r1	0.083	0.44	5.17e-4	0.001

Appendix

Table 5: Probability to observe the results of each level of semantic load at the next level for each LLM. The impact of the passage to the next level is significant for an LLM if it is below a significance threshold of 0.005.

p

-values under 0.005 are presented in bold. SR: Same Results.

	GPT-4o	o1	DeepSeek v3	DeepSeek r1
GPT-4o		0.083	0.083	0.322
o1	0.083		SR	0.159
DeepSeek v3	0.083	SR		0.16
DeepSeek r1	0.322	0.159	0.16

Appendix

Table 6: Probability to observe the results of each LLM with the other three at level 1. The impact of changing LLM is significant if it is below a significance threshold of 0.005. SR: Same Results.

	GPT-4o	o1	DeepSeek v3	DeepSeek r1
GPT-4o		0.024	SR	SR
o1	0.024		SR	0.024
DeepSeek v3	SR	SR		0.024
DeepSeek r1	SR	0.024	0.024

Appendix

Table 7: Probability to observe the results of each LLM with the other three at level 2. The impact of changing LLM is significant if it is below a significance threshold of 0.005. SR: Same Results.

	GPT-4o	o1	DeepSeek v3	DeepSeek r1
GPT-4o		0.002	0.002	SR
o1	0.002		SR	0.002
DeepSeek v3	0.002	SR		0.002
DeepSeek r1	SR	0.002	0.002

Appendix

Table 8: Probability to observe the results of each LLM with the other three at level 3. The impact of changing LLM is significant if it is below a significance threshold of 0.005.

p

-values under 0.005 are presented in bold. SR: Same Results.

	GPT-4o	o1	DeepSeek v3	DeepSeek r1
GPT-4o		6.4e-7	7.12e-6	SR
o1	6.4e-7		0.083	6.4e-7
DeepSeek v3	7.12e-6	0.083		7.12e-6
DeepSeek r1	SR	6.4e-7	7.12e-6

Appendix

Table 9: Probability to observe the results of each LLM with the other three at level 4a. The impact of changing LLM is significant if it is below a significance threshold of 0.005.

p

-values under 0.005 are presented in bold. SR: Same Results.

	GPT-4o	o1	DeepSeek v3	DeepSeek r1
GPT-4o		8.29e-10	1.13e-7	SR
o1	8.29e-10		0.024	8.29e-10
DeepSeek v3	1.13e-7	0.024		1.13e-7
DeepSeek r1	SR	8.29e-10	1.13e-7

Appendix

Table 10: Probability to observe the results of each LLM with the other three at level 4b. The impact of changing LLM is significant if it is below a significance threshold of 0.005.

p

-values under 0.005 are presented in bold. SR: Same Results.

Figure 5: Example of o3 answer to one of our sentences. While o3 correctly chooses to compute the addition, it does not generate the correct result.

Language Model Reasoning Evaluation

See all 93 papers

When Do Large Language Models Exhibit Unsolicited Deception?
Mar 31, 2025Samuel M. Taylor, Benjamin K. BergenLanguage Model Safety Evaluation Deception in Language Models
Department of Cognitive Science University of California, San Diego
Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs
Aug 31, 2026Ramya Keerthy Thatikonda, Wray Buntine, Ehsan ShareghiLogical Reasoning LLM Evaluation
Department of Data Science & AI, Monash University · ∃Department of Computer Science, University College London
Tracing the ongoing emergence of human-like reasoning in Large Language Models
May 20, 2026Paolo Morosi, Nikoleta Pantelidou, Fritz Günther +2Logical Reasoning Pragmatic Reasoning in Language Models
Departament de Filologia Catalana, Universitat Autònoma de Barcelona, Barcelona 08193, Spain. · Institut für Psychologie, Humboldt-Universitat zu Berlin, Berlin 10099, Germany. · Institució Catalana de Recerca i Estudis Avançats (ICREA), Barcelona 08010, Spain

Mar 31, 2025J/K move · Enter open · S save

When Do Large Language Models Exhibit Unsolicited Deception?

Samuel M. Taylor, Benjamin K. Bergen

Department of Cognitive Science University of California, San Diego

Language Model Safety Evaluation Deception in Language Models

Open paper AskPDF arXiv

Large Language Models (LLMs) are effective at deceiving when prompted to do so. Models that demonstrate better performance on reasoning tasks are also better at prompted deception. But under what conditions do they deceive without instruction to do so? This study evaluates unsolicited deception produced by LLMs in a preregistered experimental protocol using tools from signaling theory. We evaluated a range of 18 proprietary closed-source and open-source LLMs using modified 2x2 games (in the style of the Prisoner's Dilemma) augmented with a phase in which they can freely communicate to the other agent using unconstrained language. This setup creates an opportunity to misrepresent its actions in conditions that vary in how useful doing so might be towards goal satisfaction. The results indicate that 1) all tested LLMs misrepresent their actions in at least some conditions, 2) they are generally more likely to do so in situations in which deception is beneficial, and 3) models exhibiting better reasoning capacity overall tend to misrepresent at higher rates. Taken together, these results suggest a correlational relationship between model reasoning performance and situational deception, and reveal certain contextual factors that affect whether LLMs will misrepresent actions or not in a novel experimental configuration.

Abstract

Language Model Reasoning Evaluation

Explore similar work

When Do Large Language Models Exhibit Unsolicited Deception?

Beyond Surface Forms: Symbolic Edits as a Test for Logical Reasoning with LLMs

Tracing the ongoing emergence of human-like reasoning in Large Language Models