cs.AIFeb 26, 2026
SaveDistilling Metacognitive Behaviors through Agentic Editing
Organizations: Dept. of ECE, Seoul Nat’l Univ., Seoul, Korea · AI Center, Samsung Elec., Korea · AIIS, ASRI, INMC, ISRC, Interdisc. Prog. in AI, Seoul Nat’l Univ., Seoul, Korea
Abstract
Distilling long reasoning traces from a teacher model into a smaller student also passes on the teacher's reasoning habits. Prior work and our annotation of teacher traces show that current reasoning models often start without a plan, verify superficially, abandon promising solution paths, and take unchecked shortcuts. We propose Metacognitive Behavioral Tuning (MBT), which inserts seven metacognitive behaviors into teacher traces at the points where the reasoning lacks them and then trains the student on the edited traces. Because teacher traces are long, how the edits are made matters. Rewriting a whole trace at once summarizes long traces, and choosing all insertion points at once crowds them at the start and end. Our agentic method, MBT-A, edits the trace file line by line and records each edit as a git commit, which avoids summarizing long traces and spreads the insertions. On competition math, a Qwen3.5-4B-Base student trained on MBT-A traces scores 85.4 against 83.1 for SFT on the unedited traces and also outperforms a token-matched control and five prior methods. MBT-A improves two other students as well, and the inserted behaviors clearly transfer to the students' reasoning, with rare ones several times more frequent.
Figures & tables
Figure 1: Undesirable behaviors in 200 diagnostic teacher traces (50 problems, four each). Shortcuts and overthinking are more frequent in incorrect traces, while almost no trace lacks substantive verification.
Figure 2: MBT inserts target behaviors into teacher traces and trains the student on the edited traces. The three editing strategies share the editor model and instructions and differ in how the edits are made (the bottom row shows the MBT-A loop and one real insertion).
| Target behavior | Undesirable behaviors it addresses | Orig. (%) |
| Subgoal setting | no initial plan | 10.0 |
| Logical check | missing logical check, expected-answer guessing, unverified recall | 77.0 |
| Substantive verification | superficial verification, no substantive verification, unverified conclusion, expected-answer guessing, unverified recall, recall attempt | 96.0 |
| Targeted backtracking | shallow backtracking | 2.5 |
| Resuming abandoned paths | abandoning promising paths | 76.5 |
| Self-correction | failure to self-correct, unverified conclusion | 64.0 |
Table 1: Target behaviors, the undesirable behaviors each addresses, and the share of 200 original teacher traces that already contain it. Where each is inserted is shown in Figure 2 , and definitions are in Table A2 .
Figure 3: (a) Share of insertions in each tenth of the original reasoning (axis capped at 20%). MBT-I places only 20.8% of its insertions in the shaded middle, against 31.7% for MBT-A and 50% under uniform placement. The legend gives the KS statistic against uniform placement (lower is more even). (b) Mean length ratio by original length. MBT-R shrinks long traces. (c) MBT-R fails more often on long traces.
| AIME | HMMT | |||||
| Method | 25 | 26 | Feb25 | Feb26 | Nov25 | Avg. |
| Base model | 84.2 | 84.6 | 78.3 | 69.3 | 73.3 | 77.8 |
| Unedited SFT | 85.4 | 84.6 | 83.8 | 78.4 | 83.8 | 83.1 |
| Token-matched control | 84.6 | 83.8 | 85.8 | 77.3 | 85.0 | 83.2 |
| TI | 81.2 | 81.2 | 67.5 | 62.9 | 71.7 | 72.7 |
| ASTRO | 82.1 | 85.0 | 81.7 | 75.8 | 85.0 | 81.8 |
Table 2: Competition math with Qwen3.5-4B-Base (avg@8, %). Avg. is over all 153 problems. TI is applied to the Base model without training. Bold and underline mark the best and second best per column.
| Method | GPQA | LCB | MuSiQue | HotpotQA | 2Wiki |
| Base model | 72.0 | 47.1 | 63.0 | 88.8 | 83.2 |
| Unedited SFT | 77.5 | 47.5 | 74.3 | 91.3 | 85.4 |
| MBT-R | 77.5 | 43.5 | 69.3 | 90.1 | 83.7 |
| MBT-I | 79.2 | 48.0 | 74.8 | 91.3 | 85.7 |
| MBT-A | 78.1 | 50.3 | 74.6 | 91.4 | 86.0 |
| Items | 198 | 131 | 2,417 | 7,405 | 12,576 |
Table 3: Untrained domains (accuracy, %), with avg@8 for GPQA Diamond and LiveCodeBench (LCB) and a single sample for multi-hop QA (2Wiki is 2WikiMultihopQA). Bold and underline mark the best and second best per column.
Figure 4: Share of traces with each target behavior in 200 teacher traces before (gray) and after (blue) MBT-A editing (left), and in 200 traces each of the Unedited SFT (gray) and MBT-A (blue) students (right).
Figure 5: Odds ratio of a correct answer when each behavior is present, within the same problem and condition (unadjusted 95% CI). Only the logical check is significant after Holm correction.
Appendix figures & tables35 assets
Supplementary material from the paper’s appendix.
Appendix
| Undesirable behavior | Evidence | Reported finding | Target behavior that addresses it |
| No initial plan | Bogdan et al. (2025) , Marjanović et al. (2026) | When sentences are resampled one at a time, planning sentences are among those with the largest influence on the final answer. Models plan while executing instead of planning first. | subgoal setting |
| Missing logical check | Chen and Wang (2026) | Reflections that check the logic within a deduction are strongly associated with correct traces and much rarer in failed ones. Most reflections (78.8%) occur in planning steps rather than deductive steps. | logical check |
| Superficial verification | Chen and Wang (2026) , Stechly et al. (2025) | Most reflections in failed traces are superficial verifications that cannot catch errors. When models verify and revise their own answers, performance drops, and revision helps only when a sound external verifier gives the verdicts. | substantive verification |
| Shallow backtracking | Chen and Wang (2026) | Most backtracking in failed traces jumps back less than a tenth of the trace, a local rewind rather than a return to an earlier premise. | targeted backtracking |
| Abandoning promising paths | Wang et al. (2025c) , Lu et al. (2025) | Many incorrect answers contain a correct thought at least once. Training on traces revised to continue the abandoned paths makes traces shorter and accuracy higher. | resuming abandoned paths |
| Failure to self-correct | Tsui (2026) , Tyen et al. (2024) | Errors that models correct when given as external input are largely left uncorrected in their own outputs. Fixing succeeds given the error location. | self-correction |
Table A1: Undesirable behaviors in the traces of reasoning models reported by prior work, and the target behaviors that address them.
| Target behavior | What it does, and where | Undesirable behaviors it addresses | Original traces (%) |
| Subgoal setting | Before solving, states the goal and the subgoals in order, only in traces that start without a plan. | no initial plan | 10.0 |
| Logical check | Right after a deductive step, tests a premise the step rests on, or names a case its split does not cover. | missing logical check, expected-answer guessing, unverified recall | 77.0 |
| Substantive verification | Checks a result by a procedure that would expose an error, such as substitution, small cases, or another derivation. | superficial verification, no substantive verification, unverified conclusion, expected-answer guessing, unverified recall, recall attempt | 96.0 |
| Targeted backtracking | When a path fails, names the earlier point where the problem began and returns there. | shallow backtracking | 2.5 |
| Resuming abandoned paths | Picks up a promising idea that was raised and then dropped without being used. | abandoning promising paths | 76.5 |
| Self-correction | Finds an actual error in an earlier step and appends the correction after it, leaving the step in place. | failure to self-correct, unverified conclusion | 64.0 |
Table A2: The seven target behaviors. The last column is the share of 200 original teacher traces (100 correct, 100 incorrect) that contain the behavior at least once.
| Behavior | Definition | Correct traces (%) | Incorrect traces (%) |
| Undesirable behaviors | |||
| Recall attempt | Tries to recall the source or a known answer of the problem. | 54.3 | 67.4 |
| Unverified recall (sub-behavior of recall attempt) | Asserts a recalled source or answer specifically without checking it. | 18.1 | 34.7 |
| Expected-answer guessing | Chooses an answer by guessing what the problem setter or grader expects instead of using mathematical grounds. | 58.1 | 76.8 |
| Repeating the same interpretation issue | Revisits an interpretation issue, such as the meaning of a symbol, three or more times without new information. | 72.4 | 78.9 |
| Hedged final answer | Gives two or more different answers, or a conditional answer, as the final answer. | 20.0 | 20.0 |
Table B1: All items of the undesirable-behavior annotation scheme on the 200 diagnostic teacher traces (105 correct, 95 incorrect). Bold items appear in Figure 1 , the last two as 100 minus the rate.
Figure B1: All items of the undesirable-behavior annotation scheme on the 200 diagnostic teacher traces. Dark labels mark the items of Figure 1 .
Figure B2: A teacher trace in which undesirable behaviors lead to a wrong answer (verbatim excerpts, with labels and notes by the authors). The teacher fixes on the lower bound 10, drops the correct construction, and verifies only by rereading its own case split.
| Comparison | Median kappa |
| Agreement with the reference (40 traces) | |
| claude-opus-5.5 | 0.73 |
| gpt-6.1-sol (used) | 0.64 |
| gpt-5.6-luna | 0.55 |
| claude-sonnet-5.5 | 0.54 |
| gpt-6-luna | 0.49 |
Table B2: Annotator reliability, as the median per-behavior Cohen’s kappa under the undesirable-behavior scheme. The agreement of claude-opus-5.5 uses the 29 traces it annotates.
Figure B3: Agreement of the candidate annotators with the reference annotation (Table B2 ). The dashed line is the agreement of gpt-6.1-sol with itself.
Figure C1: Editing instructions, part 1 (verbatim excerpts), covering the situation, the purpose of the edits, and the rules that apply to every edit.
Figure C2: Editing instructions, part 2 (verbatim excerpts), showing the logical-check entry and the work steps.
Figure C3: Strategy-specific formats, namely the four commit subjects of MBT-A , the output of MBT-R , which rewrites the whole trace, and the output of MBT-I , one block per insertion.
| Condition | Traces | Tokens | vs. original |
| Unedited SFT | 995 | 22,821,876 | 1.000 |
| MBT-A | 995 | 24,432,120 | 1.071 |
| MBT-I | 995 | 23,483,410 | 1.029 |
| MBT-R | 773 | 7,557,835 | 0.331 |
| Token-matched control | 1,057 | 24,490,718 | 1.073 |
| ASTRO | 992 | 22,988,634 | 1.007 |
Table D1: Training data per condition (tokens under the Qwen3.5-4B chat template). Deletion conditions and the subsampled control are relative to the 993 traces of the deletion conditions (22,688,991 tokens). Counts below 1,000 reflect failed edits or generations and traces over 131,072 tokens.
| Use | Source |
| Competition math | NVIDIA NeMo-Skills ( NVIDIA, 2026 ) , judge/math.yaml |
| GPQA Diamond | Evaluation code of Humanity’s Last Exam ( Center for AI Safety et al., 2026 ) , centerforaisafety/hle commit 22ed3074, JUDGE_PROMPT in hle_eval/run_judge_results.py |
| MuSiQue, HotpotQA, 2WikiMultihopQA | Grader template of SimpleQA Verified ( Haas et al., 2025 ) , SIMPLEQA_VERIFIED_GRADER_TEMPLATE in src/inspect_evals/simpleqa/grader_templates.py , UKGovernmentBEIS/inspect_evals commit dbd3dd25 |
Table D2: Sources of the grading prompts, each sent as the user message to the LLM judge.
| Item | Value |
| Framework | TRL ( von Werra et al., 2020 ) , Megatron, accelerate |
| Optimizer | Muon ( Jordan et al., 2024 ; Liu et al., 2025 ) with AdamW, betas 0.9 and 0.95, eps 1e-8 |
| Learning rate | 1.0e-5, constant with 40 warmup steps |
| Batch size | 64 |
| Epochs | 15 |
| Precision | bf16, fp32 master weights |
Table D3: Training hyperparameters, shared by all students and conditions.
| Component | Version or setting |
| Training | Python 3.12.13, torch 2.14.0, transformers 5.17.0, trl 1.14.0, accelerate 1.15.0, liger-kernel 0.8.3, triton 3.8.0 |
| Serving | vllm 0.30.0, torch 2.13.0+cu130, flashinfer-python 0.6.18.post1 |
| Editing harness | Claude Code 2.1.283, E2B sandbox |
| LLM judge | gpt-5.6-luna, reasoning effort medium |
| Annotator | gpt-6.1-sol, reasoning effort medium |
| Teacher | DeepSeek-V4.1-Flash, reasoning effort max, temperature 1.0, top_p 0.95, one sample per training problem |
Table D4: Software versions and model call settings.
Figure D1: Prompt for teacher generation and for the math evaluation of the students.
| Artifact | Use | License or terms |
| Data | ||
| Nemotron-Math-v2 | training problems | CC BY 4.0 (AoPS subset), CC BY-SA 4.0 (StackExchange subsets) |
| MathArena AIME 2025, AIME 2026, HMMT Feb 2025, Feb 2026, Nov 2025 | evaluation | CC BY-NC-SA 4.0 |
| GPQA Diamond | evaluation | CC BY 4.0, examples not to be shown online |
| LiveCodeBench v6 | evaluation | CC (data), MIT (code) |
| MuSiQue | evaluation | CC BY 4.0 |
Table D5: Licenses and terms of the artifacts used in this work, checked on the official pages on October 12, 2026.
| Method | Original paper | Our implementation | Training tokens (vs. original) |
| ASTRO | Linearizes a search-tree path into one solution, inserting reflection and backtracking phrases where the search returns from an incorrect branch, and a phrase that the solution is on track where it proceeds well. | At the positions where MBT-A inserts passages, except self-correction positions, inserts the phrases of ASTRO instead. The blank part of a phrase (the point to return to) is filled by DeepSeek-V4.1-Flash. | 1.007 |
| CoT-Bridge | Supplements data with a Qwen2.5-Math-7B model trained to find and fill missing intermediate steps between adjacent steps. | Gives the paper’s zero-shot baseline prompt to DeepSeek-V4.1-Flash to find and insert missing steps between paragraphs of the reasoning section. It inserts 1,209 steps into 529 of the 1,000 traces (about 2.3 per edited trace). | 1.010 |
| RefAug | Appends a “Reflection” section (an alternative solution and a follow-up problem), written by GPT-4-turbo, which is used only in training. | Gives the paper’s prompt to DeepSeek-V4.1-Flash to append a reflection section after the response section. The reasoning section is unchanged, and at evaluation the output is cut before “Reflection:”. | 1.049 |
| Gandhi et al. | Has Claude 3.5 Sonnet solve problems anew with a system prompt asking for the four cognitive behaviors, without filtering by correctness. | Has the teacher solve each problem again with the system prompt from the paper’s repository adapted to math. The line describing the Countdown task is replaced by the math problem, and the answer-format instruction and the HINT line are removed, while the behavior instructions and five examples are kept. New answers are used without filtering, and correctness under math_verify changes for 96 of 998 problems (9.6%). | 1.049 |
| TI (Thinking Intervention) | Without training, inserts a first-person instruction where reasoning begins and lets the model continue. | At evaluation, inserts a first-person sentence asking for the behaviors at the start of the Base model’s thinking, e.g., “Okay, the user asked for this. I should … plan the steps, … catch and correct any mistake, …” | none |
Table E1: Reimplementation of prior methods. The four training-based methods use the same problems, students, and training settings as MBT, and all except Gandhi et al. start from the same original traces.
Figure E1: Text inserted by MBT-A and by the ASTRO condition at the same two positions of the same teacher trace (verbatim). ASTRO’s fixed phrases do not state what was checked.
| MBT-R | MBT-I | MBT-A | |
| Completion | |||
| Edited successfully (of 1,000) | 776 | 1,000 | 1,000 |
| Attempts per trace (mean) | 4.24 | 1.01 | 1.00 |
| Length ratio | |||
| Median | 1.000 | 1.048 | 1.112 |
| Mean | 0.864 | 1.138 | 1.271 |
Table F1: Editing statistics on the same 1,000 teacher traces. Length statistics of MBT-R use its 776 successful edits, and positions are relative to the original reasoning section, from 0 (start) to 1 (end).
Figure F1: Insertions per behavior, with the count (left) and the median position in the original reasoning (right).
| Failure reason (last call) | Traces | Share |
| No section headers (solves the problem instead of returning the file) | 156 | 69.6% |
| Output limit hit five times in a row | 44 | 19.6% |
| Copies the instructions after the last line of the file | 13 | 5.8% |
| Wrong section order or a duplicated section | 5 | 2.2% |
| Replaces original text with an omission phrase | 5 | 2.2% |
| Request timeout | 1 | 0.4% |
Table F2: Why MBT-R fails on 224 traces.
| Quantity | Median | Mean | Max | Total |
| Commits | 15 | 17.0 | 54 | 16,993 |
| Agent turns | 52 | 56.4 | 169 | 56,401 |
| Time (s) | 872 | 888 | 2,380 | 887,964 |
| Cost (USD) | 0.147 | 0.149 | 0.392 | 149.26 |
| Input tokens, cache miss | 87.3k | 97.7k | 330.6k | 97.7M |
| Input tokens, cache hit | 8.89M | 10.86M | 52.17M | 10.86B |
Table F3: Work of MBT-A per trace over 1,000 traces. An agent turn is one call of the agent to the editor model, and cost is at the official DeepSeek API prices at the time of the runs.
| Type of repair | Commits | Share |
| Sentence polishing | 3,311 | 82.4% |
| Content correction | 397 | 9.9% |
| Notation | 203 | 5.1% |
| Relocation | 107 | 2.7% |
Table F4: Commits in which the MBT-A agent repairs its own earlier insertions (4,018 commits in 711 traces), classified by keywords in the commit subjects.
Figure F2: Why MBT-R fails. Most failed outputs solve the problem instead of returning the edited trace.
Figure G1: (a) Training loss of the main Unedited SFT and MBT-A runs, averaged per epoch. (b) Math score (avg@8 on the 153 problems) at the three evaluated checkpoints.
| Condition | ep5 | ep10 | ep15 |
| Unedited SFT | 75.7 | 82.8 | 83.1 |
| MBT-A | 72.9 | 84.8 | 85.4 |
| Token-matched control | 74.8 | 83.0 | 83.2 |
| MBT-I | 74.3 | 83.7 | 83.8 |
| MBT-R | 64.1 | 76.0 | 73.9 |
| ASTRO | 72.8 | 83.7 | 81.8 |
Table G1: Math avg@8 (%) on the 153 problems per epoch. Unedited SFT and MBT-A are the runs of Table 2 , and the others are single runs. Bold and underline mark the best and second best per column.
| AIME | HMMT | |||||
| Model / condition | 25 | 26 | Feb25 | Feb26 | Nov25 | Avg. |
| MiniCPM5-2B-Base | ||||||
| Base model | 27.5 | 31.2 | 17.5 | 21.2 | 23.3 | 24.1 |
| Unedited SFT | 41.2 | 34.6 | 20.8 | 23.1 | 23.3 | 28.5 |
| MBT-A | 37.5 | 37.9 | 27.9 | 25.4 | 28.8 | 31.4 |
| Granite 4.1 3B Base | ||||||
Table G2: Competition math (avg@8, %) for the other students, trained with the same data and settings. Bold marks the better of Unedited SFT and MBT-A . a Queried in plain text without a chat template.
| Condition | Run | ep5 | ep10 | ep15 |
| Unedited SFT | run 1 ∗ | 75.7 | 82.8 | 83.1 |
| run 2 | 74.8 | 82.8 | 83.7 | |
| run 3 | 75.2 | 82.8 | 82.8 | |
| MBT-A | run 1 ∗ | 72.9 | 84.8 | 85.4 |
| run 2 | 71.9 | 84.8 | 85.0 | |
| run 3 | 71.6 | 83.4 | 85.4 |
Table G3: Three runs with different seeds (math avg@8, %). ∗ Runs reported in the main text. Bold and underline mark the best and second best per column.
Figure G2: Final math scores of the three seeds of Unedited SFT and MBT-A .
Figure G3: Competition math for the other students (Table G2 ). MBT-A raises the average of both models, though not every benchmark.
Figure H1: Share (%) of 200 student traces in which each target behavior occurs, for all ten conditions. The three MBT strategies (right of the white line) raise the behaviors that are rare under the other conditions.
Figure H2: Two students on AIME 2026, Problem 16 in MathArena (verbatim excerpts, labels and notes by the authors). The Unedited SFT student uses a wrong formula for the 10th term and checks only its addition. The MBT-A student plans, checks that every candidate difference works, fixes its earlier 10th terms, and rechecks its sum by regrouping.
| Behavior | Odds ratio [95% CI] | Raw | Holm | Discordant pairs |
| Subgoal setting | 0.89 [0.62, 1.29] | 0.53 | 1 | 116 |
| Logical check | 2.60 [1.38, 4.91] | 0.0031 | 0.022 | 50 |
| Substantive verification | 7.62 [1.30, 44.72] | 0.024 | 0.15 | 14 |
| Targeted backtracking | 1.07 [0.74, 1.54] | 0.74 | 1 | 116 |
| Resuming abandoned paths | 0.93 [0.65, 1.32] | 0.67 | 1 | 133 |
| Self-correction | 1.31 [0.65, 2.63] | 0.45 | 1 | 39 |
Table H1: Association of each behavior with correctness within the same problem and condition (437 pairs). Bold marks Holm-adjusted .
| Category | Text deleted (target behavior) |
| Problem understanding | Restating the problem, listing conditions, settling the meaning of the question, and remarks about the request itself such as the answer format or length |
| Planning and decomposition ∗ | Saying what to do before doing it, subgoals, previews of the next step, and plans for writing the final answer ( subgoal setting ) |
| Strategy selection and justification | Listing methods, choosing one, and justifying the choice. The solution by the chosen method is kept |
| Progress monitoring | Summaries of what has been obtained and what remains. Judgments that the problem is hard or one is stuck |
| Error detection and correction ∗ | (a) An incorrect step together with the text that catches it, leaving only the corrected solution. (b) Sentences saying that a path is abandoned or that one goes back, keeping the abandoned path’s work ( self-correction, targeted backtracking ) |
| Verification ∗ | All text that rechecks a result already obtained, such as substitution, recomputation, another method, special cases, premise checks, and verdicts such as “Yes.” ( substantive verification, logical check ) |
Table I1: The eight deletion categories, with the corresponding target behaviors in parentheses. Delete-4 removes the four categories marked with ∗ , and Delete-8 removes all eight.
More in this topic
Efficient Language Model Reasoning
Efficient language model reasoning studies how to improve problem-solving accuracy while reducing inference, memory, latency, and training costs.
See all 370 papers