RECAST: REconstruction of Chain-of-Thought via Auditable Structured Traces

Qi Wang; Kang Xu; Chengcheng Wan; Jiangtao Wang

cs.AIOct 30, 2026

RECAST: REconstruction of Chain-of-Thought via Auditable Structured Traces

Authors: Qi Wang, Kang Xu, Chengcheng Wan, Jiangtao Wang

Organizations: East China Normal University · Shanghai Innovation Institute · Software Engineering Institute, ECNU

Abstract

Large closed-source reasoning models expose only their final answers. The intermediate reasoning that produces those answers is hidden behind API surfaces, tool-call policies, and safety filters. Existing chain-of-thought (CoT) extraction methods assume a side channel: a visible reasoning field, a scratchpad carried through tool calls, or a signed/encrypted reasoning block. We reproduce three representative state-of-the-art extraction methods and find that, on Claude-Sonnet-5.5 accessed through the standard chat-completions interface, all three return zero CoT tokens. The failure is structural rather than a bug: the reasoning field is absent, named tool selection is rejected, free-text scratchpad arguments are filtered, and no replayable reasoning block is exposed. We therefore take a different route. We present RECAST, a reconstruction-based framework that conditions a decoder on the question and the target model's final answer, reconstructs a first-person structured rationale, cleans it, and builds SFT-ready data. To the best of our knowledge, RECAST is the first system to reconstruct CoT from frontier closed models such as Claude-Sonnet-5.5, GPT-6.1, and Gemini 3 without using hidden reasoning channels. We instantiate RECAST on Claude-Sonnet-5.5 and additionally obtain successful qualitative reconstructions on GPT-6.1 and Gemini 3 with the same answer-only protocol. The system is open-source and packaged as an agent skill: one prompt deploys it and runs the complete pipeline to produce SFT-ready CoT data. This preprint describes the system, its threat model, and a controlled artifact validation; it does not claim large-scale benchmark improvements.

Figures & tables

Method	Required side channel	Observed on Claude-Sonnet-5.5	CoT tokens
P3 Forced-Reasoning ( Luo et al., 2026 )	Named/any tool_choice plus a free-text tool argument	Named tool choice rejected with HTTP 400; free-text scratchpad arguments blocked with content_filter	0
P1 EchoCoT ( Qu et al., 2026 )	Scratchpad tool state across multiple turns	First scratchpad call blocked with content_filter ; tool invocation rate 0%	0
P2 Stealing Reasoning Traces ( Panfilov et al., 2026 )	Signed/encrypted reasoning block to replay	No signature , encrypted_content , or reasoning_text ; source billed 9,949 completion tokens; extraction error 1.0	0
Recast (ours)	Final answer $y$ plus an external decoder $D$	Only the answer is required; no hidden channel	structured trace

Table 1: Controlled reproduction of three representative CoT extraction methods on Claude-Sonnet-5.5 through a standard chat-completions interface. All three obtain zero CoT tokens. The table reports the observed interface behavior, not a benchmark score; runs use one or two questions and are intended as mechanism checks.

Panel of Figure 1 — Figure 1: The Recast pipeline. The target model is treated as a black box; only its final answer enters the reconstruction stage. The decoder produces a structured first-person trace, the cleaner removes dialogue/meta artifacts, and the dataset builder attaches a canonical answer and validates the result.

Pattern	Profile	Structure and purpose
math_solver	math	Teacher reads the problem; student derives and verifies. For proof problems, the verifier attacks the argument for gaps.
socratic	logic, planning	Repeated questions on goal, constraints, assumptions, and counterexamples before solving.
expert_panel	science	Multiple expert roles decompose the problem, check assumptions, and converge on a solution.
pair_programming	code	Driver–navigator pair writes code, tests, and counterexamples.
red_blue	secondary	Blue team proposes a solution; red team attacks edge cases and counterexamples; a judge converges.
retrospective	open-ended	Solve first, then use a retrospective/PDCA cycle to surface requirements and extensions.

Table 2: Elicitation patterns and their intended profiles. The pattern controls how the target answer is elicited; it does not control the decoder or the cleaner.

Status	Meaning and downstream handling
ok	Cleaned trace passed JSON parsing, leak checks, and length checks; used as the supervision target.
still_leaky	Dialogue or meta terms remain after retries; raw trace is used and the leak is recorded.
rejected_short	Cleaned trace is shorter than the minimum ratio; raw trace is used and the record is flagged.
rejected_long	Cleaned trace exceeds the maximum ratio (possible duplication); raw trace is used and the record is flagged.
api_error	Cleaner call failed; raw trace is used and the error is recorded.

Table 3: Cleaner outcomes and how the dataset builder handles them. A non- ok outcome never silently replaces the raw trace with a shorter or cleaner-looking one.

Target	Assistant content
cot_answer	Cleaned CoT followed by the final answer (default).
cot_only	Cleaned CoT only; the answer is kept in metadata and not trained on.
source_only	The target model’s original answer; used as an answer-only baseline.
cot_plus_source	Cleaned CoT plus the original answer; used to test whether answer style adds signal beyond the reconstructed rationale.

Table 4: Supervision target variants produced by the dataset builder.

Stage	Time (s)
Collect target answer	12.8
Decode reconstructed CoT	9.9
Clean trace	8.4
Extract/canonicalize answer	2.2
Build + validate SFT data	0.2

Table 5: Single-item artifact validation. Times are wall-clock seconds for one question; the run is a pipeline smoke test, not a benchmark.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Dataset	Profile	Primary	Secondary
MATH-500, GSM8K, AIME	math	math_solver	red_blue
HumanEval, MBPP, LiveCodeBench	code	pair_programming	red_blue
GPQA, ScienceQA	science	expert_panel	red_blue
BBH, LogiQA	logic	socratic	expert_panel
PlanBench, agent tasks	planning	socratic	retrospective
Open-ended writing	open_ended	retrospective	socratic

Appendix

Table 6: Representative routing decisions. Unknown datasets fall back to keyword scoring and then to general .

Structured Reasoning in Language Models

See all 64 papers

Aug 20, 2026

EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models

Hidden chain-of-thought (CoT) traces, especially those from frontier proprietary large reasoning models (LRMs), are valuable model assets. Yet whether these hidden CoTs can be directly extracted from black-box models remains largely unexplored. In this work, we systematically study whether hidden CoTs can be extracted near-verbatim from black-box LRMs through API interactions. We identify a previously overlooked reasoning replay surface between tool calls and develop EchoCoT, a multi-step attack that iteratively extracts hidden CoTs using API-returned fidelity signals. We further develop an LLM-based optimization framework that automatically searches for an effective universal injection trajectory across various datasets. We evaluate EchoCoT on three open-source and five frontier proprietary LRMs. On open-source LRMs, EchoCoT achieves up to 66.4% near-verbatim extraction success, with the extracted trace length within 10% of the target and at least 90% of tokens exactly matching the target CoT. The same injection trajectory also generalizes to unseen datasets, achieving up to 80% extraction success under the same criterion. For tested frontier proprietary LRMs, a substantial fraction of extracted CoTs closely align with provider-reported reasoning lengths and available CoT summaries. EchoCoT can also extract very long CoTs: on Gemini-2.5, it extracts 33,463 tokens from a 32,948-token target. These results establish hidden-CoT extraction as a practical security risk and highlight the need to better protect hidden CoT assets.

Yiting Qu, Ziqing Yang, Chi Cui +3

CISPA Helmholtz Center for Information Security

LLM Security Model Extraction Attacks Large Reasoning Models Prompt Injection Attacks on LLMs

Ask PDF

Sep 22, 2026

Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

The rapid capability gains of frontier language models are widely attributed to improved reasoning abilities, yet this cannot be verified as raw CoT traces in closed-source systems are hidden. By registering a simple custom tool through a standard API feature, we induce frontier models to externalize intermediate reasoning. Because these traces may reflect post-hoc rationalization rather than genuine reasoning, we first evaluate against native CoT on open-source models and extend to closed-source frontier models including GPT-6 Astra. We find that the extracted reasoning matches native reasoning performance and substantially outperforms no-reasoning baselines, across competition mathematics, science, and code generation. We then characterize how frontier models structure their intermediate reasoning. Across token efficiency, reasoning-step types, and induced reasoning trees, we identify systematic differences in how models externalize, compress, and organize reasoning. We find that Astra exhibits token-efficient directed reasoning, selecting a correct trajectory earlier, while resolving elementary steps internally and externalizing only crucial reasoning. These findings provide a behavioral lens on frontier-model reasoning beyond benchmark scores.

Xiaoyu Luo, Tao Ren, Wenrui Yu +3

Department of Computer Science Aalborg University, Copenhagen, Denmark · Department of Electronic Systems Aalborg University, Copenhagen, Denmark · Seafill

LLM Interpretability CoT Reasoning Efficient Language Model Reasoning Structured Reasoning Implicit Reasoning in Language Models Structured Reasoning in Language Models

Ask PDF

May 14, 2026

Stateful Reasoning via Insight Replay

Chain-of-Thought (CoT) reasoning has become a foundation for eliciting multi-step reasoning in large language models, but recent studies show that its benefits do not scale monotonically with chain length: while longer CoT generally enables a model to tackle harder problems, on a given problem, accuracy typically increases with CoT length up to a point, after which it declines. We identify a major cause of this phenomenon: as the CoT grows, the model's attention to critical insights produced earlier in the trace gradually weakens, making those insights progressively less accessible when they are most needed. Therefore, we propose InsightReplay, a stateful reasoning approach in which the model periodically extracts critical insights from its reasoning trace and replays them near the active generation frontier, keeping them accessible as the reasoning scales. Extensive experiments on a

\mathbf{2}\!\times\!\mathbf{3}\!\times\!\mathbf{4}

benchmark grid, covering model scales

\{\text{8B}, \text{30B}\}

, model families

\{\text{Qwen3.5}, \text{DeepSeek-R1-Distill-Qwen}, \text{Gemma-4}\}

, and reasoning benchmarks

\{\text{AIME}, \text{HMMT}, \text{GPQA Diamond}, \text{LiveCodeBench v5}\}

, show that 3-round InsightReplay yields accuracy gains across all 24 settings, with an averaged improvement of

\mathbf{+1.65}

points over standard CoT, and a largest single-setting gain of

\mathbf{+9.2}

points on R1-Distill-32B's LiveCodeBench v5 subset. Our results suggest that the effectiveness of test-time scaling depends not only on how much a model reasons, but also on whether critical intermediate insights remain accessible throughout long reasoning trajectories.

Bin Lei, Caiwen Ding, Jiachen Yang +2

University of Minnesota · 2Simular AI

Memory-Augmented Language Models CoT Reasoning Test-Time Scaling LLM Reasoning

Ask PDF

Abstract

Structured Reasoning in Language Models

Explore similar work

EchoCoT: Extracting Hidden Chain-of-Thought from Large Reasoning Models

Capable yet Parsimonious: Extracting and Characterizing Hidden Chain-of-Thought in Frontier Models

Stateful Reasoning via Insight Replay