Distilling Metacognitive Behaviors through Agentic Editing
Authors: Ik-hwan Kim, Hyeongrok Han, Mingi Jung, Sangwon Yu, Jinseok Hong, Sang Hun Kim, Yoonyoung Choi, Sungroh Yoon
Organizations: Dept. of ECE, Seoul Nat’l Univ., Seoul, Korea · AI Center, Samsung Elec., Korea · AIIS, ASRI, INMC, ISRC, Interdisc. Prog. in AI, Seoul Nat’l Univ., Seoul, Korea
Distilling long reasoning traces from a teacher model into a smaller student also passes on the teacher's reasoning habits. Prior work and our annotation of teacher traces show that current reasoning models often start without a plan, verify superficially, abandon promising solution paths, and take unchecked shortcuts. We propose Metacognitive Behavioral Tuning (MBT), which inserts seven metacognitive behaviors into teacher traces at the points where the reasoning lacks them and then trains the student on the edited traces. Because teacher traces are long, how the edits are made matters. Rewriting a whole trace at once summarizes long traces, and choosing all insertion points at once crowds them at the start and end. Our agentic method, MBT-A, edits the trace file line by line and records each edit as a git commit, which avoids summarizing long traces and spreads the insertions. On competition math, a Qwen3.5-4B-Base student trained on MBT-A traces scores 85.4 against 83.1 for SFT on the unedited traces and also outperforms a token-matched control and five prior methods. MBT-A improves two other students as well, and the inserted behaviors clearly transfer to the students' reasoning, with rare ones several times more frequent.
Figures & tables
Figure 1: Undesirable behaviors in 200 diagnostic teacher traces (50 problems, four each). Shortcuts and overthinking are more frequent in incorrect traces, while almost no trace lacks substantive verification.
Figure 2: MBT inserts target behaviors into teacher traces and trains the student on the edited traces. The three editing strategies share the editor model and instructions and differ in how the edits are made (the bottom row shows the MBT-A loop and one real insertion).
Table 1: Target behaviors, the undesirable behaviors each addresses, and the share of 200 original teacher traces that already contain it. Where each is inserted is shown in Figure 2 , and definitions are in Table A2 .
Figure 3: (a) Share of insertions in each tenth of the original reasoning (axis capped at 20%). MBT-I places only 20.8% of its insertions in the shaded middle, against 31.7% for MBT-A and 50% under uniform placement. The legend gives the KS statistic D against uniform placement (lower is more even). (b) Mean length ratio by original length. MBT-R shrinks long traces. (c) MBT-R fails more often on long traces.
AIME
HMMT
Method
25
26
Feb25
Feb26
Nov25
Avg.
Base model
84.2
84.6
78.3
69.3
73.3
77.8
Unedited SFT
85.4
84.6
83.8
78.4
83.8
83.1
Token-matched control
84.6
83.8
85.8
77.3
85.0
83.2
TI
81.2
81.2
67.5
62.9
71.7
72.7
ASTRO
82.1
85.0
81.7
75.8
85.0
81.8
Table 2: Competition math with Qwen3.5-4B-Base (avg@8, %). Avg. is over all 153 problems. TI is applied to the Base model without training. Bold and underline mark the best and second best per column.
Method
GPQA
LCB
MuSiQue
HotpotQA
2Wiki
Base model
72.0
47.1
63.0
88.8
83.2
Unedited SFT
77.5
47.5
74.3
91.3
85.4
MBT-R
77.5
43.5
69.3
90.1
83.7
MBT-I
79.2
48.0
74.8
91.3
85.7
MBT-A
78.1
50.3
74.6
91.4
86.0
Items
198
131
2,417
7,405
12,576
Table 3: Untrained domains (accuracy, %), with avg@8 for GPQA Diamond and LiveCodeBench (LCB) and a single sample for multi-hop QA (2Wiki is 2WikiMultihopQA). Bold and underline mark the best and second best per column.
Figure 4: Share of traces with each target behavior in 200 teacher traces before (gray) and after (blue) MBT-A editing (left), and in 200 traces each of the Unedited SFT (gray) and MBT-A (blue) students (right).
Figure 5: Odds ratio of a correct answer when each behavior is present, within the same problem and condition (unadjusted 95% CI). Only the logical check is significant after Holm correction.
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
Undesirable behavior
Evidence
Reported finding
Target behavior that addresses it
No initial plan
Bogdan et al. (2025) , Marjanović et al. (2026)
When sentences are resampled one at a time, planning sentences are among those with the largest influence on the final answer. Models plan while executing instead of planning first.
subgoal setting
Missing logical check
Chen and Wang (2026)
Reflections that check the logic within a deduction are strongly associated with correct traces and much rarer in failed ones. Most reflections (78.8%) occur in planning steps rather than deductive steps.
logical check
Superficial verification
Chen and Wang (2026) , Stechly et al. (2025)
Most reflections in failed traces are superficial verifications that cannot catch errors. When models verify and revise their own answers, performance drops, and revision helps only when a sound external verifier gives the verdicts.
substantive verification
Shallow backtracking
Chen and Wang (2026)
Most backtracking in failed traces jumps back less than a tenth of the trace, a local rewind rather than a return to an earlier premise.
targeted backtracking
Abandoning promising paths
Wang et al. (2025c) , Lu et al. (2025)
Many incorrect answers contain a correct thought at least once. Training on traces revised to continue the abandoned paths makes traces shorter and accuracy higher.
resuming abandoned paths
Failure to self-correct
Tsui (2026) , Tyen et al. (2024)
Errors that models correct when given as external input are largely left uncorrected in their own outputs. Fixing succeeds given the error location.
self-correction
Appendix
Table A1: Undesirable behaviors in the traces of reasoning models reported by prior work, and the target behaviors that address them.
Target behavior
What it does, and where
Undesirable behaviors it addresses
Original traces (%)
Subgoal setting
Before solving, states the goal and the subgoals in order, only in traces that start without a plan.
no initial plan
10.0
Logical check
Right after a deductive step, tests a premise the step rests on, or names a case its split does not cover.
When a path fails, names the earlier point where the problem began and returns there.
shallow backtracking
2.5
Resuming abandoned paths
Picks up a promising idea that was raised and then dropped without being used.
abandoning promising paths
76.5
Self-correction
Finds an actual error in an earlier step and appends the correction after it, leaving the step in place.
failure to self-correct, unverified conclusion
64.0
Appendix
Table A2: The seven target behaviors. The last column is the share of 200 original teacher traces (100 correct, 100 incorrect) that contain the behavior at least once.
Behavior
Definition
Correct traces (%)
Incorrect traces (%)
Undesirable behaviors
Recall attempt
Tries to recall the source or a known answer of the problem.
54.3
67.4
Unverified recall (sub-behavior of recall attempt)
Asserts a recalled source or answer specifically without checking it.
18.1
34.7
Expected-answer guessing
Chooses an answer by guessing what the problem setter or grader expects instead of using mathematical grounds.
58.1
76.8
Repeating the same interpretation issue
Revisits an interpretation issue, such as the meaning of a symbol, three or more times without new information.
72.4
78.9
Hedged final answer
Gives two or more different answers, or a conditional answer, as the final answer.
20.0
20.0
Appendix
Table B1: All items of the undesirable-behavior annotation scheme on the 200 diagnostic teacher traces (105 correct, 95 incorrect). Bold items appear in Figure 1 , the last two as 100 minus the rate.
Figure B1: All items of the undesirable-behavior annotation scheme on the 200 diagnostic teacher traces. Dark labels mark the items of Figure 1 .
Figure B2: A teacher trace in which undesirable behaviors lead to a wrong answer (verbatim excerpts, with labels and notes by the authors). The teacher fixes on the lower bound 10, drops the correct construction, and verifies only by rereading its own case split.
Comparison
Median kappa
Agreement with the reference (40 traces)
claude-opus-5.5
0.73
gpt-6.1-sol (used)
0.64
gpt-5.6-luna
0.55
claude-sonnet-5.5
0.54
gpt-6-luna
0.49
Appendix
Table B2: Annotator reliability, as the median per-behavior Cohen’s kappa under the undesirable-behavior scheme. The agreement of claude-opus-5.5 uses the 29 traces it annotates.
Figure B3: Agreement of the candidate annotators with the reference annotation (Table B2 ). The dashed line is the agreement of gpt-6.1-sol with itself.
Figure C1: Editing instructions, part 1 (verbatim excerpts), covering the situation, the purpose of the edits, and the rules that apply to every edit.
Figure C2: Editing instructions, part 2 (verbatim excerpts), showing the logical-check entry and the work steps.
Figure C3: Strategy-specific formats, namely the four commit subjects of MBT-A , the output of MBT-R , which rewrites the whole trace, and the output of MBT-I , one block per insertion.
Condition
Traces
Tokens
vs. original
Unedited SFT
995
22,821,876
1.000
MBT-A
995
24,432,120
1.071
MBT-I
995
23,483,410
1.029
MBT-R
773
7,557,835
0.331
Token-matched control
1,057
24,490,718
1.073
ASTRO
992
22,988,634
1.007
Appendix
Table D1: Training data per condition (tokens under the Qwen3.5-4B chat template). Deletion conditions and the subsampled control are relative to the 993 traces of the deletion conditions (22,688,991 tokens). Counts below 1,000 reflect failed edits or generations and traces over 131,072 tokens.
Evaluation code of Humanity’s Last Exam ( Center for AI Safety et al., 2026 ) , centerforaisafety/hle commit 22ed3074, JUDGE_PROMPT in hle_eval/run_judge_results.py
MuSiQue, HotpotQA, 2WikiMultihopQA
Grader template of SimpleQA Verified ( Haas et al., 2025 ) , SIMPLEQA_VERIFIED_GRADER_TEMPLATE in src/inspect_evals/simpleqa/grader_templates.py , UKGovernmentBEIS/inspect_evals commit dbd3dd25
Appendix
Table D2: Sources of the grading prompts, each sent as the user message to the LLM judge.
Item
Value
Framework
TRL ( von Werra et al., 2020 ) , Megatron, accelerate
Optimizer
Muon ( Jordan et al., 2024 ; Liu et al., 2025 ) with AdamW, betas 0.9 and 0.95, eps 1e-8
Learning rate
1.0e-5, constant with 40 warmup steps
Batch size
64
Epochs
15
Precision
bf16, fp32 master weights
Appendix
Table D3: Training hyperparameters, shared by all students and conditions.
DeepSeek-V4.1-Flash, reasoning effort max, temperature 1.0, top_p 0.95, one sample per training problem
Appendix
Table D4: Software versions and model call settings.
Figure D1: Prompt for teacher generation and for the math evaluation of the students.
Artifact
Use
License or terms
Data
Nemotron-Math-v2
training problems
CC BY 4.0 (AoPS subset), CC BY-SA 4.0 (StackExchange subsets)
MathArena AIME 2025, AIME 2026, HMMT Feb 2025, Feb 2026, Nov 2025
evaluation
CC BY-NC-SA 4.0
GPQA Diamond
evaluation
CC BY 4.0, examples not to be shown online
LiveCodeBench v6
evaluation
CC (data), MIT (code)
MuSiQue
evaluation
CC BY 4.0
Appendix
Table D5: Licenses and terms of the artifacts used in this work, checked on the official pages on October 12, 2026.
Method
Original paper
Our implementation
Training tokens (vs. original)
ASTRO
Linearizes a search-tree path into one solution, inserting reflection and backtracking phrases where the search returns from an incorrect branch, and a phrase that the solution is on track where it proceeds well.
At the positions where MBT-A inserts passages, except self-correction positions, inserts the phrases of ASTRO instead. The blank part of a phrase (the point to return to) is filled by DeepSeek-V4.1-Flash.
1.007
CoT-Bridge
Supplements data with a Qwen2.5-Math-7B model trained to find and fill missing intermediate steps between adjacent steps.
Gives the paper’s zero-shot baseline prompt to DeepSeek-V4.1-Flash to find and insert missing steps between paragraphs of the reasoning section. It inserts 1,209 steps into 529 of the 1,000 traces (about 2.3 per edited trace).
1.010
RefAug
Appends a “Reflection” section (an alternative solution and a follow-up problem), written by GPT-4-turbo, which is used only in training.
Gives the paper’s prompt to DeepSeek-V4.1-Flash to append a reflection section after the response section. The reasoning section is unchanged, and at evaluation the output is cut before “Reflection:”.
1.049
Gandhi et al.
Has Claude 3.5 Sonnet solve problems anew with a system prompt asking for the four cognitive behaviors, without filtering by correctness.
Has the teacher solve each problem again with the system prompt from the paper’s repository adapted to math. The line describing the Countdown task is replaced by the math problem, and the answer-format instruction and the HINT line are removed, while the behavior instructions and five examples are kept. New answers are used without filtering, and correctness under math_verify changes for 96 of 998 problems (9.6%).
1.049
TI (Thinking Intervention)
Without training, inserts a first-person instruction where reasoning begins and lets the model continue.
At evaluation, inserts a first-person sentence asking for the behaviors at the start of the Base model’s thinking, e.g., “Okay, the user asked for this. I should … plan the steps, … catch and correct any mistake, …”
none
Appendix
Table E1: Reimplementation of prior methods. The four training-based methods use the same problems, students, and training settings as MBT, and all except Gandhi et al. start from the same original traces.
Figure E1: Text inserted by MBT-A and by the ASTRO condition at the same two positions of the same teacher trace (verbatim). ASTRO’s fixed phrases do not state what was checked.
MBT-R
MBT-I
MBT-A
Completion
Edited successfully (of 1,000)
776
1,000
1,000
Attempts per trace (mean)
4.24
1.01
1.00
Length ratio ρ
Median
1.000
1.048
1.112
Mean
0.864
1.138
1.271
Appendix
Table F1: Editing statistics on the same 1,000 teacher traces. Length statistics of MBT-R use its 776 successful edits, and positions are relative to the original reasoning section, from 0 (start) to 1 (end).
Figure F1: Insertions per behavior, with the count (left) and the median position in the original reasoning (right).
Failure reason (last call)
Traces
Share
No section headers (solves the problem instead of returning the file)
156
69.6%
Output limit hit five times in a row
44
19.6%
Copies the instructions after the last line of the file
13
5.8%
Wrong section order or a duplicated section
5
2.2%
Replaces original text with an omission phrase
5
2.2%
Request timeout
1
0.4%
Appendix
Table F2: Why MBT-R fails on 224 traces.
Quantity
Median
Mean
Max
Total
Commits
15
17.0
54
16,993
Agent turns
52
56.4
169
56,401
Time (s)
872
888
2,380
887,964
Cost (USD)
0.147
0.149
0.392
149.26
Input tokens, cache miss
87.3k
97.7k
330.6k
97.7M
Input tokens, cache hit
8.89M
10.86M
52.17M
10.86B
Appendix
Table F3: Work of MBT-A per trace over 1,000 traces. An agent turn is one call of the agent to the editor model, and cost is at the official DeepSeek API prices at the time of the runs.
Type of repair
Commits
Share
Sentence polishing
3,311
82.4%
Content correction
397
9.9%
Notation
203
5.1%
Relocation
107
2.7%
Appendix
Table F4: Commits in which the MBT-A agent repairs its own earlier insertions (4,018 commits in 711 traces), classified by keywords in the commit subjects.
Figure F2: Why MBT-R fails. Most failed outputs solve the problem instead of returning the edited trace.
Figure G1: (a) Training loss of the main Unedited SFT and MBT-A runs, averaged per epoch. (b) Math score (avg@8 on the 153 problems) at the three evaluated checkpoints.
Condition
ep5
ep10
ep15
Unedited SFT
75.7
82.8
83.1
MBT-A
72.9
84.8
85.4
Token-matched control
74.8
83.0
83.2
MBT-I
74.3
83.7
83.8
MBT-R
64.1
76.0
73.9
ASTRO
72.8
83.7
81.8
Appendix
Table G1: Math avg@8 (%) on the 153 problems per epoch. Unedited SFT and MBT-A are the runs of Table 2 , and the others are single runs. Bold and underline mark the best and second best per column.
AIME
HMMT
Model / condition
25
26
Feb25
Feb26
Nov25
Avg.
MiniCPM5-2B-Base
Base model
27.5
31.2
17.5
21.2
23.3
24.1
Unedited SFT
41.2
34.6
20.8
23.1
23.3
28.5
MBT-A
37.5
37.9
27.9
25.4
28.8
31.4
Granite 4.1 3B Base
Appendix
Table G2: Competition math (avg@8, %) for the other students, trained with the same data and settings. Bold marks the better of Unedited SFT and MBT-A . a Queried in plain text without a chat template.
Condition
Run
ep5
ep10
ep15
Unedited SFT
run 1 ∗
75.7
82.8
83.1
run 2
74.8
82.8
83.7
run 3
75.2
82.8
82.8
MBT-A
run 1 ∗
72.9
84.8
85.4
run 2
71.9
84.8
85.0
run 3
71.6
83.4
85.4
Appendix
Table G3: Three runs with different seeds (math avg@8, %). ∗ Runs reported in the main text. Bold and underline mark the best and second best per column.
Figure G2: Final math scores of the three seeds of Unedited SFT and MBT-A .
Figure G3: Competition math for the other students (Table G2 ). MBT-A raises the average of both models, though not every benchmark.
Figure H1: Share (%) of 200 student traces in which each target behavior occurs, for all ten conditions. The three MBT strategies (right of the white line) raise the behaviors that are rare under the other conditions.
Figure H2: Two students on AIME 2026, Problem 16 in MathArena (verbatim excerpts, labels and notes by the authors). The Unedited SFT student uses a wrong formula for the 10th term and checks only its addition. The MBT-A student plans, checks that every candidate difference works, fixes its earlier 10th terms, and rechecks its sum by regrouping.
Behavior
Odds ratio [95% CI]
Raw p
Holm p
Discordant pairs
Subgoal setting
0.89 [0.62, 1.29]
0.53
1
116
Logical check
2.60 [1.38, 4.91]
0.0031
0.022
50
Substantive verification
7.62 [1.30, 44.72]
0.024
0.15
14
Targeted backtracking
1.07 [0.74, 1.54]
0.74
1
116
Resuming abandoned paths
0.93 [0.65, 1.32]
0.67
1
133
Self-correction
1.31 [0.65, 2.63]
0.45
1
39
Appendix
Table H1: Association of each behavior with correctness within the same problem and condition (437 pairs). Bold marks Holm-adjusted p<0.05 .
Category
Text deleted (target behavior)
Problem understanding
Restating the problem, listing conditions, settling the meaning of the question, and remarks about the request itself such as the answer format or length
Planning and decomposition ∗
Saying what to do before doing it, subgoals, previews of the next step, and plans for writing the final answer ( subgoal setting )
Strategy selection and justification
Listing methods, choosing one, and justifying the choice. The solution by the chosen method is kept
Progress monitoring
Summaries of what has been obtained and what remains. Judgments that the problem is hard or one is stuck
Error detection and correction ∗
(a) An incorrect step together with the text that catches it, leaving only the corrected solution. (b) Sentences saying that a path is abandoned or that one goes back, keeping the abandoned path’s work ( self-correction, targeted backtracking )
Verification ∗
All text that rechecks a result already obtained, such as substitution, recomputation, another method, special cases, premise checks, and verdicts such as “Yes.” ( substantive verification, logical check )
Appendix
Table I1: The eight deletion categories, with the corresponding target behaviors in parentheses. Delete-4 removes the four categories marked with ∗ , and Delete-8 removes all eight.
Large Language Models (LLMs) demonstrate strong reasoning performance, yet their ability to reliably monitor, diagnose, and correct their own errors remains limited. We introduce a psychologically grounded metacognitive framework that operationalizes Ann Brown's regulatory cycle (Planning, Monitoring, and Evaluation) as a structured prompting architecture, and study its integration within a lightweight dual-process MetaController for adaptive effort allocation. Across diverse reasoning and diagnostic benchmarks (GSM8K, CRUXEval, MBPP, AIME, CorrectBench, and TruthfulQA) using Llama-3 and Qwen-3 (8B), explicit regulatory structuring substantially improves error diagnosis and yields a threefold increase in successful self-correction. Blinded human evaluations over 580 query pairs show an 84% aggregate preference for trustworthiness and metacognitive self-awareness over standard and Chain-of-Thought baselines. Grounding LLM reasoning in established cognitive theory offers a principled path toward more transparent and diagnostically robust AI systems.
Abraham Paul Elenjical, Vivek Hruday Kavuri, Vasudeva Varma
Large reasoning models improve performance on challenging problems by allocating additional computation before answering, but longer reasoning does not always lead to better results and can introduce substantial redundant reasoning on simple problems. Conversely, aggressively shortening reasoning can degrade performance on difficult ones. Effective reasoning therefore requires dynamically deciding when additional computation is useful based on the reasoner's capabilities and evolving solution state. Existing approaches often rely on predefined budgets or intervention rules, retrain the target reasoner, or require additional supervision. We introduce MetaCtrl, a lightweight controller that adaptively regulates a frozen reasoner without predefined token budgets or reasoner retraining. We formulate reasoning regulation as a sequential metacognitive control problem: MetaCtrl observes the evolving reasoning trace and decides whether to continue, simplify, skip redundant steps, or conclude reasoning. It is trained directly with reinforcement learning using a reward that prioritizes correctness while favoring shorter trajectories among correct solutions, requiring neither supervised intervention trajectories nor problem-specific budgets. Across seven benchmarks spanning mathematics, science, and code, MetaCtrl consistently improves the accuracy of LRMs while reducing their reasoning length. On DeepSeek-R1-Distill-Qwen-7B, it improves average accuracy by 4.7 points while reducing generation length by 53.3%. Without further training, the same controller transfers to an unseen reasoner (e.g., Qwen3-14B), improving average accuracy by 2.9 points and reducing generation length by 50.3%. These results establish MetaCtrl as a plug-and-play controller for improving reasoning accuracy while substantially reducing inference-time generation. The code is available at https://github.com/binbin2xs/MetaCtrl.
Zhibin Wen, Tao Han, Lei Bai +2
Southern University of Science and Technology · The Hong Kong University of Science and Technology · Shanghai Artificial Intelligence Laboratory +1
Large language models (LLMs) often expose useful signals of self-monitoring: before solving a problem, they can estimate whether they are likely to succeed, and after solving it, they can judge whether their answer is likely to be correct. However, these signals are typically measured or elicited in isolation, rather than used to control inference. In this work, we ask whether LLMs possess latent metacognitive ability that can be turned into effective test-time control. Inspired by the Nelson--Narens theory from cognitive psychology, we propose a metacognitive harness that separates monitoring from reasoning. For each problem, the model first reports a pre-solve feeling-of-knowing (FOK) signal; after each solve attempt, it reports a post-solve judgment-of-learning (JOL) signal. Rather than treating these signals as passive confidence estimates, the harness turns them into an explicit control interface for reasoning: it decides when to trust the current solution, when to retry with compact metacognitive feedback, and when to pass multiple attempts to a final aggregator. Across text, code, and multimodal reasoning benchmarks, our harness substantially improves a fixed Claude Sonnet-4.6 base model without parameter updates or benchmark-specific fine-tuning. On the evaluated public benchmark snapshots, it raises pooled accuracy from 48.3 to 56.9 and exceeds the strongest listed leaderboard entries on the three primary evaluation settings: HLE-Verified, LiveCodeBench v6, and R-Bench-V. These results suggest that strong LLMs may already possess useful metacognitive ability, but require an explicit control harness to act on it during reasoning.