EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models

Yiting Qu; Ziqing Yang; Chi Cui; Ye Leng; Junjie Chu; Yang Zhang

cs.CRAug 20, 2026

EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models

Authors: Yiting Qu, Ziqing Yang, Chi Cui, Ye Leng, Junjie Chu, Yang Zhang

Organizations: CISPA Helmholtz Center for Information Security

Abstract

Hidden chain-of-thought (CoT) traces, especially those from frontier proprietary large reasoning models (LRMs), are valuable model assets. Yet whether these hidden CoTs can be directly extracted from black-box models remains largely unexplored. In this work, we systematically study whether hidden CoTs can be extracted near-verbatim from black-box LRMs through API interactions. We identify a previously overlooked reasoning replay surface between tool calls and develop EchoCoT, a multi-step attack that iteratively extracts hidden CoTs using API-returned fidelity signals. We further develop an LLM-based optimization framework that automatically searches for an effective universal injection trajectory across various datasets. We evaluate EchoCoT on three open-source and five frontier proprietary LRMs. On open-source LRMs, EchoCoT achieves up to 66.4% near-verbatim extraction success, with the extracted trace length within 10% of the target and at least 90% of tokens exactly matching the target CoT. The same injection trajectory also generalizes to unseen datasets, achieving up to 80% extraction success under the same criterion. For tested frontier proprietary LRMs, a substantial fraction of extracted CoTs closely align with provider-reported reasoning lengths and available CoT summaries. EchoCoT can also extract very long CoTs: on Gemini-2.5, it extracts 33,463 tokens from a 32,948-token target. These results establish hidden-CoT extraction as a practical security risk and highlight the need to better protect hidden CoT assets.

Figures & tables

Figure 1: EchoCoT extracts a 33,463-token CoT from Gemini-2.5 [ 45 ] , close to the API-reported CoT length of 32,948 tokens. Additional examples are shown in Appendix Figure 12 and Figure 13 , with more available in EchoCoT-Viewer .

Figure 2: Overview of EchoCoT, a CoT extraction attack via multi-step tool interactions.

Figure 3: Overview of the injection optimization framework in EchoCoT. The full optimization iteratively repeats this process across multiple batches.

Model	Method	Tool Inv. Rate (% $\uparrow$ )	Ans. Match Rate (% $\uparrow$ )	Length Error ( $\downarrow$ )	Textual-Level Metrics ( $\uparrow$ )			Attack Success Rate (% $\uparrow$ )
Model	Method	Tool Inv. Rate (% $\uparrow$ )	Ans. Match Rate (% $\uparrow$ )	Length Error ( $\downarrow$ )	Token F1	ROUGE-L	Token EM	ASR@99	ASR@95	ASR@90
DeepSeek-V4-Flash	Direct Prompting	–	–	1.635	0.301	0.184	0.145	0.0	0.0	0.0
	CoT Synthesis	–	–	1.441	0.353	0.215	0.160	0.0	0.0	0.0
	REP	–	88.3	0.888	0.308	0.223	0.174	0.3	0.3	0.3
	EchoCoT-Manual	93.1	91.7	0.582	0.652	0.588	0.568	23.9	34.2	36.1
	EchoCoT-LGO	93.9	91.7	0.301	0.735	0.698	0.666	37.8	46.1	50.6

Table 1: Performance of EchoCoT and baselines on the OpenThoughts test set. LGO denotes trajectories optimized for length fidelity, while LTGO denotes trajectories jointly optimized for length and textual fidelity. ASR@

x

is the fraction of samples with Length Error

\leq 1-x/100

and Token-EM

\geq x/100

. The best value for each model is highlighted in bold.

Figure 4: Example of CoT extraction from the target model DeepSeek-V4-Flash. The target and extracted CoTs contain 21,106 and 21,109 tokens, with a Token-EM of 0.999. Only three of 1,132 lines differ, highlighted in green.

Model	Method	MATH500	JEEBench	LiveCodeBench
DeepSeek	Direct Prompting	0.0	0.0	0.0
	CoT Synthesis	0.0	0.0	0.0
	REP	3.0	3.0	0.0
	EchoCoT-Manual	64.0	46.0	40.0
	EchoCoT-LGO	58.0	46.0	44.0
	EchoCoT-LTGO	80.0	71.0	64.0

Table 2: Cross-dataset transferability on three unseen datasets. We report ASR@90 (%). Full results are provided in Appendix Table 7 .

Figure 5: Extraction fidelity of EchoCoT-LTGO on DeepSeek-V4-Flash. (a) Successful extractions span target CoT lengths up to 20K+ tokens; (b) ASR@90 remains relatively high up to 16K tokens; (c) ASR@90 remains high across question sources.

Figure 6: Cumulative ASR@90 of EchoCoT-LTGO across tool-call steps. Successful extractions begin to emerge at the second tool call across all target models.

Target Model	Failed Samples	No Tool Invocation	Under - Extraction	Over - Extraction	Content Rewriting
DeepSeek	206	20.4%	45.6%	26.2%	7.8%
Qwen	481	5.2%	64.0%	20.2%	10.6%
GLM	369	8.9%	62.1%	21.7%	7.3%

Table 3: Failure-mode breakdown of EchoCoT-LTGO across the four evaluation datasets.

Target Model	CoT Sum. (%)	Tool Inv. Rate (%)	Mean Tokens		Longest Extraction		$E_{\mathrm{len}}$ (%)
Target Model	CoT Sum. (%)	Tool Inv. Rate (%)	Ext.	Target	Ext.	Target	$\boldsymbol{>0.3}$	$\boldsymbol{(0.1,0.3]}$	$\boldsymbol{\leq 0.1}$
Gemini-2.5	87.8	99.3	1,849	3,744	23,429	18,568	40.8	22.0	36.5
Gemini-3.1	100.0	100.0	774	991	3,080	2,446	38.3	29.8	32.0
Gemini-3.5	98.0	97.5	723	1,474	17,645	18,119	67.5	10.8	19.3
Sonnet-4.6	95.0	95.0	1,403	1,747	13,280	15,765	17.0	68.5	9.5
Opus-4.6	31.5	99.3	1,017	1,280	12,750	14,974	16.5	65.8	17.0

Table 4: Length statistics of extracted CoTs from frontier proprietary models. We calculate all percentages over 400 samples from the four evaluation sets, with 100 randomly sampled questions from each set.

Target Model	$N$	Entailment	$\boldsymbol{R^{sum}_{\mathrm{tok}}}$	$\boldsymbol{R^{sum}_{\mathrm{tok}}}$ (%)
Target Model	$N$	Entailment	$\boldsymbol{R^{sum}_{\mathrm{tok}}}$	$\boldsymbol{>0.629}$	$\boldsymbol{>0.743}$	$\boldsymbol{>0.857}$
Gemini-2.5	205	0.954	0.605	47.3	30.2	9.3
Gemini-3.1	247	0.929	0.435	7.3	0.4	0.0
Gemini-3.5	119	0.850	0.479	31.1	7.6	0.8
Sonnet-4.6	312	0.987	0.828	98.4	84.0	32.4
Opus-4.6	95	0.994	0.836	95.8	87.4	38.9

Table 5: Semantic and text fidelity of extracted CoTs from frontier proprietary models. We include samples with a valid CoT summary and Length Error

\leq 0.3

. Recall thresholds

0.629/0.743/0.857

correspond to estimated Token-F1

=0.7/0.8/0.9

under a linear fit shown in Appendix Figure 14 .

Figure 7: Searching behaviors and language switching in Gemini-3.5 extracted CoT.

Defense	Setting	DeepSeek	Qwen	GLM	Mean
No Defense	–	66.0	24.0	48.0	46.0
Reasoning Removal	–	0.0	0.0	0.0	0.0
System Prompt	Original	15.0	0.0	0.0	5.0
System Prompt	Adaptive	29.0	0.0	0.0	9.7
Reason Length Removal	Original	25.0	2.0	17.0	14.7
Reason Length Removal	Adaptive	49.0	5.0	37.0	30.3

Table 6: ASR@90 (%) under different defense solutions on 100 random OpenThoughts test examples. Adaptive denotes an attacker that re-optimizes its injection trajectory against the defended model. We mark the strongest practical defense under adaptive attacks in bold and the idealized upper bound underlined .

\epsilon

denotes the relative perturbation range.

Appendix figures & tables15 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Analysis of EchoCoT’s extraction and proxy fidelity signals. (a) visualizes the correlation between Summary Token Recall and ground-truth Token-F1 on three open-source models; (b) shows how the correlation between the two changes with various Length Errors. Summary Token Recall is a more reliable proxy for ground-truth extraction fidelity when Length Error is low.

Panel of Figure 9 — Figure 9: Extraction performance using different optimization objectives. We compare our two-level objective with a mean-only objective that directly optimizes the average fidelity scores. The two-level objective substantially improves Token-EM and ASR@90, especially for EchoCoT-LGO.

Panel of Figure 10 — Figure 10: Extraction performance of different optimization workflows. We compare the full Inject+Reflect+Distill pipeline with Inject-Only and Inject+Reflect . The full pipeline achieves the best overall extraction fidelity, with lower Length Error and higher Token-EM and ASR@90.

Figure 11: Extraction performance of maximum tool-call steps in EchoCoT. Increasing the maximum steps from one to three substantially reduces Length Error and improves Token-EM and ASR@90. Additional steps provide limited gains.

Panel of Figure 12 — Figure 12: Target CoT summary and the extracted CoT from Sonnet-4.6 with the target question sampled from OpenThoughts dataset. The target CoT contains 5,505 tokens, while the extracted CoT contains 4,896 tokens (Length Error=0.111, Summary Token Recall=0.805, Entailment Score=1). We use the same colors to mark matching content in both texts.

Panel of Figure 13 — Figure 13: Target CoT summary and the extracted CoT from Gemini-3.5 with the target question sampled from OpenThoughts dataset. The target CoT contains 18,119 tokens, while the extracted CoT contains 17,645 tokens (Length Error=0.026, Summary Token Recall=0.864, Entailment Score=1). We use the same colors to mark matching content in both texts.

Figure 14: Relationship between Summary Token Recall (extracted CoT vs. CoT summary) and Token-F1 (extracted CoT vs. full CoT) across 332 samples (Length Error

\leq 0.3

). The dashed line shows the linear fit

y=0.88x+0.15

(

r=0.577

p<0.001

Panel of Figure 15 — Figure 15: Prompt used for CoT summarization during the optimization process.

Panel of Figure 16 — Figure 16: System prompt used for the LLM optimizer during the CoT extraction attack. We omit details due to space limitations. Please refer to our code repository for the complete prompt.

Panel of Figure 17 — Figure 17: Prompt for the Inject stage of the LLM optimizer.

Panel of Figure 18 — Figure 18: Prompt for the Reflect stage of the LLM optimizer.

Panel of Figure 19 — Figure 19: Prompt for the Distill stage of the LLM optimizer.

Panel of Figure 20 — Figure 20: Initial experience file provided to the LLM optimizer.

Panel of Figure 21 — Figure 21: Defensive system prompt we used during the defense evaluation.

Model	Method	MATH500			JEEBench			LiveCodeBench
Model	Method	Length Error	Token EM	ASR@90	Length Error	Token EM	ASR@90	Length Error	Token EM	ASR@90
DeepSeek-V4-Flash	Direct Prompting	0.702	0.230	0.0	0.627	0.103	0.0	1.305	0.130	0.0
	CoT Synthesis	0.720	0.270	0.0	0.452	0.129	0.0	1.228	0.167	0.0
	REP	0.508	0.353	3.0	0.707	0.176	3.0	0.831	0.079	0.0
	EchoCoT-Manual	0.117	0.835	64.0	0.293	0.668	46.0	0.326	0.537	40.0
	EchoCoT-LGO	0.228	0.737	58.0	0.337	0.629	46.0	0.362	0.553	44.0

Appendix

Table 7: Cross-dataset transferability of our method and baselines on three unseen evaluation datasets: MATH500, JEEBench , and LiveCodeBench. We report Length Error, Token-EM, and ASR@90 (%) to save space.

LLM Security

Large language model (LLM) security studies attacks, privacy risks, and defenses across model training, inference, and deployment.

See all 314 papers

Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators
Aug 30, 2026Armaan Singh, Ryan Trinh Le, Jasmine Kaur +5Implicit Reasoning in Language Models Language Model Probing
1Stanford University · 2York University · 3Facebook
Stealing Reasoning Traces from Proprietary LLM APIs
Aug 10, 2026Alexander Panfilov, David Schmotz, Ilia Shumailov +5Model Extraction Attacks Adversarial Attacks
MATS Research · ELLIS Institute Tübingen · Max Planck Institute for Intelligent Systems +4
Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
Sep 30, 2026Yihuai Hong, Shauli Ravfogel, Chen Zhao +1CoT Faithfulness Faithfulness of Language Model Explanations
New York University · NYU Shanghai

Aug 30, 2026J/K move · Enter open · S save

Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

Armaan Singh, Ryan Trinh Le, Jasmine Kaur, Abdullah Sultan+4

1Stanford University · 2York University · 3Facebook

Implicit Reasoning in Language Models Language Model Probing Large Reasoning Models LLM Interpretability LLM Evaluation Language Model Reasoning Evaluation+1

Open paper AskPDF arXiv

Large language models often answer complex reasoning questions without revealing intermediate steps, raising whether they reason latently or complete patterns. We propose the Hidden CoT Detection Score (HCDS), a comparative behavioral and mechanistic signal measuring whether neutral-prompt behavior aligns more closely with explicit CoT or explicit no- CoT. Here, hidden CoT operationally denotes this neutral-prompt CoT-like alignment; HCDS does not directly observe or prove an unexposed reasoning trace. On GSM8K, HCDS is significantly positive for both Qwen3-4B variants (Thinking

+1.87

p = 1.2 \times 10^{-7}

; Instruct

+1.41

p = 1.9 \times 10^{-4}

), replicates across a different inference stack and quantization within

0.08

(

+1.80

and

+1.45

), and is not significantly positive in seven of eight length-adjusted calibration-control cells. The unadjusted score produces large positive scores on single-step arithmetic and numeric factual lookup. The variants also respond differently to no-CoT instructions: Instruct complies from the prompt alone, whereas Thinking continues reasoning and requires intervention. These findings show stronger, less prompt-conditional CoT-like behavior in the reasoning-tuned model, consistent with but not proof of latent reasoning. HCDS thus investigates latent reasoning without relying on models' self-reported traces.

Abstract

LLM Security

Explore similar work

Detecting Hidden Chain-of-Thought in Large Language Models with Linguistic, Behavioral, and Mechanistic Indicators

Stealing Reasoning Traces from Proprietary LLM APIs

Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment