cs.AIOct 15, 2026

Entertain, then Cancel: How Thinking Makes Reasoning Models Read Some and Or Literally

Authors: Zhizhuo Kou, Jian Yang, Pengfei Hao, Junyu Luo, Jingshu Peng, Sirui Han, Yike Guo

Organizations: 1The Hong Kong University of Science and Technology · 2The Hong Kong University of Science and Technology (Guangzhou) · Peking University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

When a speaker says ``some students passed'', listeners infer that not all of them did. Reasoning models now let users decide how long a model thinks before it answers, but we do not know what extra thinking does to such everyday inferences. We study this directly: we keep a model's weights fixed, change only how much it thinks, and record which reading it chooses. A second tool cuts the model's own reasoning at successive points and asks for an answer, showing where the reading changes. Thinking makes models read logical words as logic: it cancels the implicatures of some (not all) and or (not both) in most tested open reasoning models and in GPT-5, while context-driven inferences such as indirect answers improve. The trace shows why: models first entertain the enriched reading and cancel it later. Answering from that early reading, an exploratory inference-time remedy, recovers these implicatures without losing literal accuracy. Evaluations of reasoning models should measure how they interpret, not only how accurate they are.

Figures & tables

Figure 1: Thinking acts as a logic lens on logical words. some and or are read as ∃x\exists x and ∨\vee , while adequate keeps ‘not good’.
Figure 2: Thinking makes some more literal in most open reasoning models. Vermilion: counted literal shift; blue: more pragmatic ( Figure 11 ).
Figure 3: A reading probe on fixed weights. Items with a trigger (①) are answered with thinking off, capped or natural (②); we read the chosen reading RR and the log-odds LL (③), also after successive trace prefixes (④), for every model and checkpoint.
 Logical wordsRest of the batteryThree-way choice
Modelsomeormodalslexical scalesindirect ans.someor
Qwen3 (thinking off →\rightarrow natural thinking)
Qwen3-4B−- 37.2 [ −- 55.1, −- 17.9]−- 61.6 [ −- 74.7, −- 47.5]−- 32.3 [ −- 47.9, −- 15.6]−- 7.7 [ −- 11.9, −- 4.0]−- 0.4 [ −- 7.2, 6.8] ∘−- 9.9−- 11.9
Qwen3-8B−- 28.2 [ −- 41.0, −- 16.7]−- 36.4 [ −- 49.5, −- 24.2]−- 12.5 [ −- 22.9, −- 2.1]5.5 [2.4, 8.8]6.5 [ −- 1.1, 14.7] ∘−- 34.0−- 13.8
Qwen3-14B−- 21.8 [ −- 37.2, −- 9.0]−- 13.1 [ −- 21.2, −- 6.1]4.2 [ −- 3.1, 13.5] ∘6.8 [3.3, 10.8]1.4 [ −- 5.4, 8.2] ∘−- 11.4−- 2.4
Qwen3-30B-A3B−- 28.2 [ −- 39.7, −- 17.9]−- 39.4 [ −- 51.5, −- 26.3]−- 19.8 [ −- 36.5, −- 2.1]0.7 [ −- 2.6, 4.0] ∘9.5 [2.5, 16.8]−- 12.8−- 11.8
Table 1: Thinking reduces the pragmatic reading of logical words across families. Change with thinking (pp) [95% CI]; bold / underlined : largest/second absolute change per column; circ : CI includes 0.
Figure 4: The enriched reading is endorsed early in the trace and abandoned later. Pragmatic reading RR after the first tt tokens of the model’s own trace, with 95% bands; panel (d) and the lexical-scale lines come from an earlier, smaller run of the probe.
Analysis choicesomeor
Qwen3-8B
Reference: core, Yes/No, off →\rightarrow natural−- 28.2−- 36.4
Baseline: forced-empty thinking block−- 39.7−- 33.3
Budget 128, stop phrase37.24.0
Budget 512, stop phrase5.1−- 15.2
Budget 1024, stop phrase−- 21.8−- 31.3
Table 2: Full thinking lowers the reading under every analysis choice. Change in the pragmatic reading with thinking (pp); short budgets raise it before it falls.
Figure 5: Pragmatic reading rate for Qwen3-8B vs. thinking budget. Capped with stop phrase (hollow: plain cap). Lower strip: realized tokens. Bands: 95% cluster-bootstrap CIs.
 Default reading (%)
StagesomeorlexicalLUDWIG
OLMo-3-Think (native thinking; cost = NAT −- forced-empty)
base75.6 ±\pm 4.474.7 ±\pm 1.839.5 ±\pm 0.467.0 ±\pm 4.0
Think-SFT39.7 ±\pm 5.950.5 ±\pm 3.55.3 ±\pm 0.776.8 ±\pm 2.8
Think-DPO53.8 ±\pm 3.848.5 ±\pm 5.24.6 ±\pm 0.775.1 ±\pm 2.4
RL step 10047.4 ±\pm 2.251.5 ±\pm 0.05.3 ±\pm 0.775.1 ±\pm 3.4
Table 3: The literal default appears with thinking SFT in OLMo-3 and DPO in Tülu-3. Default reading (%) under a fixed 4-shot prompt, mean ±\pm s.d..
ModelMethodFelicitySpeakerPresup.Literal
Qwen3-8BNatural thinking1.258.265.787.9
 Task-aware instruction3.560.266.892.0
 ours20.562.565.787.9
DeepSeek-R1Natural thinking2.255.562.386.3
 Task-aware instruction2.657.862.287.0
 ours8.158.862.986.3
Table 4: Answering from the entertained reading restores scalar implicatures without losing literal accuracy (held-out test families; accuracy, %). Felicity: IMPPRES some / or felicity items.
Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

GroupHugging Face identifiers
Hybrid thinkingQwen/Qwen3-{0.6B,1.7B,4B,8B,14B,30B-A3B,32B} , Qwen/Qwen3-8B-Base ( Yang et al., 2025 )
Effort levelsopenai/gpt-oss-{20b,120b} ( OpenAI et al., 2025 )
OLMo-2 stagesallenai/OLMo-2-1124-7B , -SFT , -DPO , -Instruct ; 13B analogues ( Team OLMo et al., 2024 )
OLMo-3 stagesallenai/Olmo-3-1025-7B , Olmo-3-7B-Think-SFT , -Think-DPO , -Think ; RL-Zero {Math, Code, IF, General, Mix} step branches ( Team Olmo et al., 2025 )
Tülu-3 stagesallenai/Llama-3.1-Tulu-3-8B-SFT , -DPO , Llama-3.1-Tulu-3-8B , Llama-3.1-Tulu-3.1-8B ( Lambert et al., 2024 )
Distilled / zero-RLdeepseek-ai/DeepSeek-R1-Distill-Qwen-{1.5B,7B,14B} ( DeepSeek-AI, 2025 ) ; Qwen2.5 bases ( Qwen et al., 2024 ) ; SimpleRL-Zoo ( Zeng et al., 2025 ) ; Open-Reasoner-Zero ( Hu et al., 2025 )
Appendix
Table 5: Models by group. Commit hashes of every checkpoint are recorded with each run.
Measurent512 tokensnat
LUDWIG, prompts of Ruis et al. (2023) : accuracy
0-shot79.784.082.0
5-shot71.979.678.6
30-shot67.678.178.7
ImplicatureX ( Spinoso-Di Piano et al., 2026 ) : accuracy
Scalar, recognition100.089.965.2
Appendix
Table 6: On ImplicatureX, thinking lowers scalar-implicature recognition and raises cancellation, as Spinoso-Di Piano et al. (2026) report. Qwen3-8B, accuracy (%) without thinking, with a 512-token cap and with natural thinking.
  Pragmatic readingYes-bias checks
ModelDose contrastYes/No inferenceThree-way choice“Yes” without utterancePragmatic, flipped question
GPT-5minimal →\rightarrow high↓\downarrow 15.2 *↓\downarrow 8.8 *↑\uparrow 26.9 *↓\downarrow 3.4
Claude Sonnet 4.6off →\rightarrow 16k↑\uparrow 19.2 *↓\downarrow 9.7 *↑\uparrow 7.9 *↓\downarrow 13.0 *
Claude Opus 4.8off →\rightarrow high↑\uparrow 10.8 *↓\downarrow 1.3↑\uparrow 3.9↑\uparrow 5.0 *
Appendix
Table 7: Closed models: robustness of the thinking effect on logical-word items (pp, highest minus lowest dose), on items answered validly at both doses; two samples per cell; ∗ : paired 95% cluster-bootstrap CI excludes 0. “Yes” without utterance: change in the rate of answering Yes when the speaker’s sentence is removed. Flipped question: change in the pragmatic rate when the question is reworded so that No is the pragmatic answer. A pure shift towards Yes raises the first and lowers the second. In the three-way format pragmatic answers are rare for every model (0–9%), which limits how far the rate can rise. Claude baselines use a 512-token answer cap.
  Logical wordsLexical scalesContext
ModelDose contrastsomeormodalsgradable adj.scalesLUDWIG
GPT-5minimal →\rightarrow high effort↓\downarrow 20.6 ±\pm 4.4 *↓\downarrow 18.8 ±\pm 10.6 *↓\downarrow 12.5 ±\pm 3.5 *↑\uparrow 0.0 ±\pm 3.5↑\uparrow 7.5 ±\pm 1.2 *↑\uparrow 6.2 ±\pm 1.8
GPT-5.4minimal →\rightarrow high effort↑\uparrow 2.5↓\downarrow 20.0 *↓\downarrow 15.0 *↓\downarrow 7.5↑\uparrow 15.0 *↑\uparrow 2.5
o4-minilow →\rightarrow high effort↓\downarrow 1.2↓\downarrow 6.2↓\downarrow 12.5↑\uparrow 5.0↑\uparrow 5.0↓\downarrow 2.5
DeepSeek-V4-Pro ¶thinking off →\rightarrow on↑\uparrow 8.9 ±\pm 3.6 *↑\uparrow 1.9 ±\pm 6.3↑\uparrow 1.2 ±\pm 1.8↑\uparrow 8.8 ±\pm 4.8 *↑\uparrow 41.7 ±\pm 2.2 *↑\uparrow 10.0 ±\pm 3.5 *
DeepSeek-V4-Flash ¶thinking off →\rightarrow on↑\uparrow 16.9 ±\pm 2.7 *↑\uparrow 4.4 ±\pm 2.7↓\downarrow 1.2 ±\pm 1.8↑\uparrow 5.0 ±\pm 3.5↑\uparrow 10.8 ±\pm 1.2 *↑\uparrow 5.0 ±\pm 0.0
Appendix
Table 8: Closed and API-served models: change in the pragmatic reading (pp) from the lowest to the highest available thinking dose; LUDWIG is accuracy. ∗ : 95% cluster-bootstrap CI over items excludes 0. ¶ Open-weight DeepSeek-V4 served through our LiteLLM proxy; Yes/No format only. ‡ Gemini 2.5 Pro cannot switch thinking off; 128 tokens is its minimum budget. Robustness checks in Table 7 ; 80 some , 80 or and 40 modal items.
 Yes/No inferenceThree-way choice
Modelsomeorsomeor
Qwen3-4B−- 49.8 [ −- 53.9, −- 45.6]−- 53.0 [ −- 57.6, −- 48.4]−- 9.9 [ −- 12.7, −- 7.3]−- 11.9 [ −- 14.7, −- 9.2]
Qwen3-8B−- 37.7 [ −- 42.1, −- 33.3]−- 37.7 [ −- 41.6, −- 33.8]−- 34.0 [ −- 38.2, −- 29.9]−- 13.8 [ −- 17.1, −- 10.6]
Qwen3-14B−- 22.6 [ −- 27.1, −- 18.3]−- 15.9 [ −- 19.1, −- 12.7]−- 11.4 [ −- 14.4, −- 8.6]−- 2.4 [ −- 4.0, −- 1.0]
Qwen3-30B-A3B−- 35.7 [ −- 39.8, −- 31.4]−- 40.9 [ −- 45.1, −- 36.8]−- 12.8 [ −- 15.7, −- 10.1]−- 11.8 [ −- 14.3, −- 9.3]
Qwen3-32B−- 55.6 [ −- 60.1, −- 50.8]−- 52.7 [ −- 57.0, −- 48.3]−- 38.0 [ −- 42.9, −- 33.1]−- 19.7 [ −- 23.1, −- 16.3]
Appendix
Table 9: On the full held-out sets, some and or turn literal with thinking in both formats in most models. Change with thinking (pp) [95% CI], 300 items per trigger; grey: CI includes 0. Some models start near the floor in the three-way format.
Figure 6: Trace probe on further families (EXAONE-Deep-7.8B, Nemotron-Nano-9B-v2, Phi-4-mini-reasoning, Skywork-OR1-7B, Llama-3.1-Nemotron-Nano-8B): pragmatic reading RR after the first tt tokens of the model’s own trace, with 95% bands over items. The yy -axis runs from 0 to 100 (0 to 80 in Figure 4 ). In panel (e) the value at t=0t=0 is the model’s collapse to one answer under a forced-empty thinking block.
Figure 7: Trace probe on the newest generation (Qwen3.5-4B, Qwen3.5-9B, Qwen3.8-27B): pragmatic reading RR after the first tt tokens of the model’s own trace, with 95% bands over items. The early lexical-scale jump in the Qwen3.5 panels is systematic and confined to sampled answers (see text).
ModelCircaNOPE
Qwen3-4B15.3 [10.4, 20.4]7.8 [2.3, 13.1]
Qwen3-8B17.5 [12.4, 22.7]9.9 [4.5, 15.5]
Qwen3-14B24.4 [18.9, 30.0]9.6 [4.4, 14.9]
Qwen3-30B-A3B8.3 [4.0, 12.7]10.4 [5.3, 15.5]
Qwen3-32B3.1 [ −- 0.9, 7.1]9.2 [3.8, 14.4]
Qwen3.5-4B16.7 [12.9, 20.4]5.8 [1.4, 10.4]
Appendix
Table 10: Thinking raises or keeps accuracy on two further context-driven sets in every model. Change with thinking (pp) [95% CI]; grey: CI includes 0; Circa ( Louis et al., 2020 ) , NOPE ( Parrish et al., 2021 ) .
Figure 8: Thinking effect for every item family and model. Change in pragmatic reading with thinking (pp); vermilion: more literal; blue: more pragmatic; framed: 95% CI excludes 0; hatched: not run. Values in Tables 12 and 8 .
 Qwen3-8BQwen3-32B
Methodsome ↑\uparrowor ↑\uparrowLUDWIG ↑\uparrowsome ↑\uparrowor ↑\uparrowLUDWIG ↑\uparrow
No thinking ( nt )29.5 ±\pm 5.937.4 ±\pm 3.576.5 ±\pm 2.665.4 ±\pm 3.855.6 ±\pm 3.583.9 ±\pm 1.2
Natural thinking ( nat )0.0 ±\pm 0.01.0 ±\pm 1.883.5 ±\pm 1.615.4 ±\pm 3.88.1 ±\pm 3.583.5 ±\pm 1.6
Prompted CoT, thinking off a10.3 ±\pm 5.96.1 ±\pm 3.075.4 ±\pm 2.246.2 ±\pm 10.245.5 ±\pm 13.980.7 ±\pm 2.2
Seed sentence opening the thinking trace (26 tokens each)
Neutral seed2.6 ±\pm 2.21.0 ±\pm 1.877.5 ±\pm 2.210.3 ±\pm 5.96.1 ±\pm 6.184.9 ±\pm 2.2
Appendix
Table 11: The intent seed restores more enrichment than length-matched control seeds. Pragmatic reading and LUDWIG accuracy (%); mean ±\pm s.d. over seeds; bold / underlined : best/second among thinking rows.
Figure 9: Thinking makes some and or more literal in every major model family. Change with thinking (pp, 95% CI). Size trends: Figure 10 .
Figure 10: The literal shift holds across sizes and families. Thinking effect on some and or (pp, 95% CI) by model size for Qwen3, Qwen3.5 and R1-Distill-Qwen, with Qwen3.8, MiMo, openPangu and DeepSeek-R1-0528 as single points.
 Generalized implicatures: ΔR\Delta R (pp) ↑\uparrowΔ\Delta Acc. (pp) ↑\uparrow
Modelsomeormodalslexical scalesIMPPRES allLUDWIG
Qwen3, distilled from larger models (0.6B–30B-A3B) or fully RL-trained (32B); nat −- nt
Qwen3-0.6B †↓\downarrow 50.0 ±\pm 10.2 *↓\downarrow 83.8 ±\pm 4.6 *↓\downarrow 62.5 ±\pm 12.5 *↓\downarrow 70.4 ±\pm 2.0 *↓\downarrow 59.1 ±\pm 1.6 *↑\uparrow 9.8 ±\pm 3.2 *
Qwen3-1.7B↓\downarrow 10.3 ±\pm 8.9↓\downarrow 47.5 ±\pm 4.6 *↓\downarrow 33.3 ±\pm 9.5 *↓\downarrow 25.4 ±\pm 7.3 *↓\downarrow 23.6 ±\pm 1.8 *↑\uparrow 17.2 ±\pm 0.6 *
Qwen3-4B↓\downarrow 37.2 ±\pm 4.4 *↓\downarrow 61.6 ±\pm 4.6 *↓\downarrow 32.3 ±\pm 14.4 *↓\downarrow 7.7 ±\pm 2.0 *↓\downarrow 26.5 ±\pm 2.8 *↓\downarrow 0.4 ±\pm 2.7
Qwen3-8B↓\downarrow 28.2 ±\pm 5.9 *↓\downarrow 36.4 ±\pm 5.2 *↓\downarrow 12.5 ±\pm 6.2 *↑\uparrow 5.5 ±\pm 1.4 *↓\downarrow 9.9 ±\pm 2.4 *↑\uparrow 6.5 ±\pm 2.8
Appendix
Table 12: Full results: the literal shift on some and or holds in 30 of 40 open reasoning models. Change in the reading chosen from no thinking ( nt ) to natural thinking ( nat ) on the same weights, by item family, on the held-out split; for OLMo-3-Think the baseline is a forced-empty thinking block (the model has no switch), and for gpt-oss the contrast is high minus low reasoning effort. Generalized families use the inference framing: IMPPRES quantifier, connective and modal triggers ( Jeretic et al., 2020 ) and scalar-diversity lexical scales ( van Tiel et al., 2016 ; Gotzner et al., 2018 ; Pankratz and van Tiel, 2021 ) . LUDWIG: indirect answers ( Ruis et al., 2023 ) . Cells: mean ±\pm standard deviation over 3 decoding seeds; blue = more pragmatic or more accurate, vermilion = more literal; * : the 95% cluster-bootstrap confidence interval over items excludes 0 (uncorrected; the mixed-effects tests are in Section C.1 ). † Without thinking, Qwen3-0.6B accepts 86–100% of inferences (a yes-bias), so it is outside the pre-registered 10–90% range and is excluded from the dose–response analysis. Table 14 lists the absolute rates.
Hyp.PredictionOutcome (evidence)Status
H1 effortPragmatic reading on generalized implicatures rises with thinkingFalls for some or or (CI excluding 0) in 30 of 40 open models, among them EXAONE-Deep-7.8B, Phi-4-mini-reasoning, every newest-generation model (Qwen3.5 0.8B–9B, Qwen3.8-27B), QwQ-32B, Nemotron-Super-49B, R1-Distill-Llama-70B, every Qwen3 size from 1.7B, gpt-oss-20b, OLMo-3-32B-Think, Phi-4-reasoning(-plus), R1-Distill-Llama-8B, MiMo, Nemotron-Nano, AceReason and Skywork-OR1; rises in Magistral-Small (a shift towards answering Yes , Section B.3 ) Granite-3.3, Llama-3.1-Nemotron-Nano-8B (Yes/No format only; literal in the three-way choice) and openPangu-Embedded-7B-model (Yes/No format only; flat in the three-way choice); no significant change in gpt-oss-120b, OLMo-3-7B-Think, SmolLM3 and openPangu-Embedded-7B-V1.1 ( Table 12 ); GPT-5 falls in two formats ( Section B.3 )not supported
H1 default-and-cancelFalls under the felicity framings, ≥\geq 5 pp in ≥\geq 2 of 4B/8B/14BHolds for some / or per family; the pre-registered IMPPRES aggregate at 14B is flat (0.5 pp), so the aggregate gate criterion failedpartly supported
H1b capability crossoverNegative at small sizes, positive at ≥\geq 32B on ≥\geq 2 families, reproduced in a same-recipe pairQwen3-32B negative on some / or ; OLMo-3-Think 7B null and 32B positive on IMPPRES but negative on some ; no family flips sign within a pair on ≥\geq 2 familiesnot supported
H2 splitPCIs improve while GCIs fall; ≥\geq 3 PCI sets improve or stay flatAccuracy rises on Fig-QA and PUB at every Qwen3 size from 4B to 32B (8B: +10.7 and +10.1 pp), on Circa and NOPE in 14 models with no decrease ( Table 10 ), and on presupposition and LUDWIG in Qwen3-8B; the budget ×\times class interaction is significant in Qwen3-4B, 8B and 14B ( Section D.3 )supported
H2b speaker knowledgeDefault-and-cancel: thinking lowers RR in both knowledge conditions; effort/capability: thinking widens the gapKnowledge sensitivity ≈\approx 0 in Qwen3 and OLMo; in gpt-oss-120b it shrinks with effort (84.0 →\rightarrow 62.5 pp)default-and-cancel supported (one model)
H2c framing ×\times classTruth framing denies implicatures of both classes; meaning framing lets PCIs improvethe interpretation-truth arm was not runnot tested
Appendix
Table 13: Outcome of every pre-registered prediction (held-out split). “Not tested” marks predictions whose arm was not run.
 Qwen3-4BQwen3-8BQwen3-14B
PhenomenonntnatΔ\DeltantnatΔ\Delta   
Generalized implicatures: pragmatic reading RR (%) ↑\uparrow
Category sentences, ternary5.0 ±\pm 0.02.5 ±\pm 0.0↓\downarrow 2.52.5 ±\pm 0.00.0 ±\pm 0.0↓\downarrow 2.52.5 ±\pm 0.00.0 ±\pm 0.0↓\downarrow 2.5
Scalar diversity16.6 ±\pm 1.18.8 ±\pm 1.0↓\downarrow 7.74.9 ±\pm 0.410.4 ±\pm 1.0↑\uparrow 5.52.0 ±\pm 0.78.8 ±\pm 1.4↑\uparrow 6.8
IMPPRES scalar49.7 ±\pm 1.823.2 ±\pm 1.3↓\downarrow 26.522.6 ±\pm 1.812.7 ±\pm 1.2↓\downarrow 9.914.5 ±\pm 1.515.0 ±\pm 2.1↑\uparrow 0.5
Presupposition: projection (%) ↑\uparrow
Appendix
Table 14: Per-phenomenon results for Qwen3-4B, 8B and 14B. Reading chosen without ( nt ) and with natural thinking ( nat ), mean ±\pm s.d. over three seeds, held-out split; Δ\Delta in pp; –: not run in this form.
Hyp.PredictionOutcome (evidence)Status
Scenario validityHumans find some misleading only under full access ( ≥\geq 80%)100.0% of 24 (blind consensus labels)met
Open-answer codingLLM reference rater agrees with the human audit (lower bound of κ≥0.40\kappa\geq 0.40 )best rater κ\kappa = 0.57 [0.34, 0.79]; open answers reported descriptivelynot met
Bott–Noveck itemsCategory sentences move with thinkingNear the literal ceiling in every format and condition for Qwen3 4B–32B; only binary felicity at 14B movesfloor set
Appendix
Table 13 , continued (part 3 of 3).
Hyp.PredictionOutcome (evidence)Status
H3 training originThe literal shift appears at RLVR, not SFT; grows with RL stepsTülu-3: at DPO, before RLVR; OLMo-3: at thinking SFT, RL-Zero flat, and the thinking cost shrinks over RL steps ( Table 3 )mixed
H4 loadA load lowers RR without thinkingWith 2,048 words of unrelated preceding text and thinking off, RR on logical words moves from 22.6 to 14.2% in Qwen3-8B but from 14.5 to 20.9% in Qwen3-14B; no consistent directionnot supported
H5 human orderingρ≥0.3\rho\geq 0.3 with human rates per scalenot runnot tested
H6 restorableIST restores ≥\geq 50% of the shift; a length-matched neutral seed does notWith 26-token seeds, the intent seed recovers 12.8% of a 29.5% no-thinking reading on some in Qwen3-8B ( nat : 0.0%); neutral and logic seeds recover little or none ( Table 11 )partly met: specific to intent, below 50%
H7 locusThe literal preference appears inside the trace, after a markerTruncating the model’s own trace: the pragmatic reading of some first rises (Qwen3-8B, +23.1 pp after 128 tokens) and falls below the no-thinking level later in the trace ( Figure 4 ); literalness markers such as technically precede the change in only 20.6% of changing Qwen3-8B traces, and occur as often in traces that do not change (39.1%)crossing supported; marker order not supported
Appendix
Table 13 , continued (part 2 of 3).
Figure 11: Thinking reduces enrichment of some , or or both in most tested models; other inference types move differently. Change in pragmatic reading with thinking (pp, 95% CI): vermilion, mean of some and or ; blue, mean of lexical scales and indirect answers; ⧫\blacklozenge , three-way choice. Every family: Figure 8 . Q3.5-35B: Qwen3.5-35B-A3B.
PredictionMeasurement on LMsHyp.
Effort orderingRR on GCIs vs. x=log⁡2(B+1)x=\log_{2}(B+1) under felicity framings (inference; ternary and 1–7 for category sentences); true/false as the logic-mode contrastSection D.1
GCI/PCI splitbudget ×\times class interaction on RR (PCI: accuracy, literal-error share)Section D.1
Speaker knowledgeΔR\Delta R (full −- partial knowledge) at nt vs. natH2b
Framing ×\times classPCI accuracy under meaning vs. interpretation-truth framing, nt vs. natH2c
Training origindefault RR by stage (log-prob scoring); cost of thinking by stage; ρ\rho (RL step, literal rate)Section D.1
LoadRR under a memory load, crossed with thinkingSection D.1
Appendix
Table 15: From theory to measurement. Each prediction that separates the effort and default-and-cancel accounts, the quantity we measure, and the hypothesis it tests. RR : proportion of committed answers that take the pragmatic reading; BB : thinking budget in tokens.
BlockModelsItemsSeedsTTCap
E133,87730.68192
E1S132,05230.68192
E1SF53,60030.68192
E1SFX173,60030.68192
E1Q62,05230.68192
E1P72,05230.68192
Appendix
Table 16: Run settings per experiment block: models, items per model, decoding seeds, temperature and thinking cap (tokens).
More in this topic

Pragmatic Reasoning in Language Models

Pragmatic reasoning in language models studies how models infer and communicate meanings shaped by context, intent, shared knowledge, and social norms.

See all 103 papers

Explore similar work

CardsList
Mar 27, 2026

Why Models Know But Don't Say: Chain-of-Thought Faithfulness Divergence Between Thinking Tokens and Answers in Open-Weight Reasoning Models

Extended-thinking models expose a second text-generation channel ("thinking tokens") alongside the user-visible answer. This study examines 12 open-weight reasoning models on MMLU and GPQA questions paired with misleading hints. Among the 10,506 cases where models actually followed the hint (choosing the hint's target over the ground truth), each case is classified by whether the model acknowledges the hint in its thinking tokens, its answer text, both, or neither. In 55.4% of these cases the model's thinking tokens contain hint-related keywords that the visible answer omits entirely, a pattern termed thinking-answer divergence. The reverse (answer-only acknowledgment) is near-zero (0.5%), confirming that the asymmetry is directional. Hint type shapes the pattern sharply: sycophancy is the most transparent hint, with 58.8% of sycophancy-influenced cases acknowledging the professor's authority in both channels, while consistency (72.2%) and unethical (62.7%) hints are dominated by thinking-only acknowledgment. Models also vary widely, from near-total divergence (Step-3.5-Flash: 94.7%) to relative transparency (Qwen3.5-27B: 19.6%). These results show that answer-text-only monitoring misses more than half of all hint-influenced reasoning and that thinking-token access, while necessary, still leaves 11.8% of cases with no verbalized acknowledgment in either channel.

Richard J. Young

University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA

CoT FaithfulnessLarge Reasoning ModelsFaithfulness of Language Model Explanations
AskPDF
Oct 14, 2026

Selective Reasoning for Typed Decision Models: When to Start, Read, and Stop

Typed decision models need an interface that can use additional reasoning while retaining candidate outputs. We train a shared candidate-answer reader on a frozen Qwen3.5-4B backbone to unify direct decisions with on-demand, interruptible reasoning. The reader returns distributions from empty and partial reasoning prefixes. A benefit gate starts reasoning, a natural-state trigger requests explicit reads, and an entropy rule accepts acquired answers and cancels generation. Measurements distinguish answer-query work, generated output, and pipeline time. Across 768 questions from ProofWriter, LogiQA 2.0, and ARC-Challenge, accuracy rises from 70.96% for direct reading to 84.38%. Relative to 512-token terminal reading, accuracy improves by 7.29 percentage points with 2.25% more mean output work. Relative to 1,024-token terminal reading, pipeline time falls by 46.30% with an accuracy difference of -1.17 points. Exact-prefix controls isolate the saving from interruption, while read-scheduling controls distinguish reduced observation work from shorter response time.

Bin Zheng, Zheng Chen

University of Electronic Science and Technology of China

Efficient Language Model ReasoningLLM Decision-MakingLLM InferenceLogical Reasoning
AskPDF
Oct 15, 2026

Are Large Reasoning Models Good Everyday Reasoners? A Comparison with Non-Reasoning Models on Logical Reasoning Tasks

Large reasoning models (LRMs) that leverage test-time computation achieve strong performance on mathematical and scientific benchmarks and are increasingly deployed in real-world applications. However, their ability to perform everyday logical reasoning, which often departs from textbook formal logic, remains underexplored. In this work, we systematically evaluate how well reasoning models align with everyday reasoning patterns by introducing a benchmark based on linguistic and pragmatic reasoning phenomena. We also compare reasoning models with non-reasoning models under controlled prompting conditions. We find that, in many cases, (1) reasoning models show degraded performance in everyday reasoning when such reasoning deviates from formal logical rules, (2) some non-reasoning models with chain-of-thought prompting outperform comparable reasoning models in everyday reasoning, and (3) prompting with additional instructions provides limited control over reasoning behavior. These findings suggest that current reasoning models do not reliably capture everyday reasoning and point to limitations in post-training, motivating improved training objectives and evaluation across diverse forms of reasoning.

Kentaro Ozeki, Risako Ando, Takanobu Morishita +3

Keio University

Language Model Reasoning EvaluationLLM ReasoningLarge Reasoning ModelsPragmatic Reasoning in Language Models
AskPDF