Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping

Zhenyu Lei; Qiong Wu; Jianxiong Dong; Yinhan He; Emily Dodwell; Yushun Dong; Jundong Li

cs.CLMar 6, 2026

Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping

Authors: Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong, Jundong Li

Organizations: University of Virginia · AT&T Chief Data Office · Florida State University

Abstract

Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability. Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors. We introduce Reasoning Editing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents a fundamental trade-off between Generality, the ability of an edit to generalize across different tasks sharing the same reasoning pattern, and Locality, the ability to preserve other reasoning capabilities. Through systematic investigation, we uncover the Circuit-Interference Law: Edit interference between reasoning patterns is proportional to the overlap of their neural circuits. Guided by this principle, we propose REdit, the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping, which directly addresses the generality-locality trade-off by disentangling overlapping circuits; (ii) Meta-Contrastive Learning, which extends transferability to novel reasoning patterns; and (iii) Dual-Level Protection, which preserves preexisting abilities by constraining reshaping update directions and regularizing task-level predictions. Extensive experiments with Qwen-2.5-3B on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consistently achieves superior generality and locality compared to baselines, with additional validation in mathematics showing broader potential. Our code is available at https://github.com/LzyFischer/REdit.

Figures & tables

Panel of Figure 1 — Figure 1: LLM reasoning deficiencies and editing trade-off.

Figure 2: Correlation between circuit distance and interference. (a–c) Scatter plots with regression lines show that larger distances consistently correspond to reduced interference across different distance metrics. (d) Density plots of Pearson correlations confirm consistent negative associations.

Figure 3: Editing success rates across methods on ContextHub. REdit achieves success rates comparable to other approaches, confirming that it does not compromise the model’s fundamental editing capabilities.

Dataset	Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	Ours
Level 1	Generality	$60.7~{\pm~2.3}$	$\underline{72.2}~{\pm 1.4}$	$63.8~{\pm~2.9}$	$67.8~{\pm 3.2}$	$67.9~{\pm 1.9}$	$\textbf{74.1}~{\pm~1.6}$
Level 1	Locality	N/A	$61.5~{\pm 0.7}$	$84.9~{\pm~1.6}$	$\underline{89.8}~{\pm 3.1}$	$87.0~{\pm 0.9}$	$\textbf{94.3}~{\pm~0.4}$
Level 2	Generality	$53.2~{\pm~1.4}$	$\underline{63.6}~{\pm 2.9}$	$58.4~{\pm~0.1}$	$61.3~{\pm 1.1}$	$58.8~{\pm 1.5}$	$\textbf{64.8}~{\pm~1.2}$
Level 2	Locality	N/A	$59.4~{\pm 4.1}$	$91.5~{\pm~0.0}$	$93.1~{\pm 0.1}$	$\underline{93.3}~{\pm 0.0}$	$\textbf{94.3}~{\pm~0.5}$
Level 3	Generality	$45.1~{\pm~1.6}$	$52.6~{\pm 0.4}$	$50.1~{\pm~0.8}$	$51.5~{\pm 3.3}$	$\underline{54.2}~{\pm 0.8}$	$\textbf{55.0}~{\pm~1.6}$
Level 3	Locality	N/A	$52.3~{\pm 1.0}$	$92.3~{\pm~2.8}$	$\textbf{94.6}~{\pm 2.7}$	$92.2~{\pm 0.7}$	$\underline{94.4}~{\pm~0.8}$

Table 1: Main results on ContextHub evaluated with generality and locality metrics. The best and second-best scores are highlighted in bold and underlined , respectively. Raw denotes the performance of the unedited LLM. For BIMT, we apply the same LoRA-based editing method as in REdit.

Dataset	Metric	Raw	w/o MCL	w/o NSP	w/o PDP	Ours
Level 1	Generality	$60.7~{\pm~2.3}$	${72.9}~{\pm~0.4}$	$73.3~{\pm~0.2}$	$\underline{73.4}~{\pm~0.5}$	$\textbf{74.1}~{\pm~1.6}$
Level 1	Locality	N/A	$\underline{90.7}~{\pm~1.8}$	$89.5~{\pm~0.3}$	$90.1~{\pm~2.5}$	$\textbf{94.3}~{\pm~0.4}$
Level 2	Generality	$53.2~{\pm~1.4}$	$\underline{62.5}~{\pm~0.3}$	$62.4~{\pm~1.6}$	$61.3~{\pm~2.0}$	$\textbf{64.8}~{\pm~1.2}$
Level 2	Locality	N/A	$\textbf{94.9}~{\pm~0.6}$	$93.0~{\pm~1.8}$	$94.0~{\pm~0.8}$	$\underline{94.3}~{\pm~0.5}$
Level 3	Generality	$45.1~{\pm~1.6}$	$\underline{53.8}~{\pm~1.3}$	$50.9~{\pm~0.6}$	$51.8~{\pm~0.6}$	$\textbf{55.0}~{\pm~1.6}$
Level 3	Locality	N/A	$\underline{93.7}~{\pm~1.3}$	$92.8~{\pm~1.1}$	$92.8~{\pm~1.2}$	$\textbf{94.4}~{\pm~0.8}$

Table 2: Ablation studies on ContextHub evaluated with generality and locality metrics. The best and second-best scores are highlighted in bold and underlined , respectively.

Figure 4: Performance on unseen reasoning patterns after circuit reshaping with different ratios for training. REdit consistently outperforms baselines without reshaping.

Panel of Figure 5 — Figure 5: Circuit–interference relationship before and after circuit reshaping. (a,b) Scatter plots of intra- and inter-pattern measurements show improved separability in interference and circuit distance. (c) Silhouette scores across reasoning patterns indicate consistent gains in cluster separation.

Figure 6: Evaluation on mathematical reasoning benchmark TemplateGSM.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Abstract Instance	Contextual Instance
$(\texttt{vxkgr}\lor\texttt{caunc})\to\texttt{ybyz}$ . Given ybyz is False, what is the value of caunc ?	If an area of land has experienced significant uplift or been shaped by powerful erosional forces, then the terrain will feature tall, steep mountains. Given that the area does not have tall, steep mountains, can it be determined if powerful erosional forces have shaped the land?

Appendix

Table 3: Level-1 example instantiations in ContextHub.

Math Template	Instantiated Problem
[NAME] sold [NUM1] [ITEM] to [her/his/their] friends in April at a [LOCATION] in [COUNTY] , [STATE] . In May, [PRONOUN] sold [NUM2] [ITEM] . How many [ITEM] did [NAME] sell altogether in April and May?	Rosy Plascencia sold 238 air fryers to her friends in April at a yoga studio boutique in Bracken County, Kentucky. In May, they sold 119 air fryers. How many air fryers did Rosy Plascencia sell altogether in April and May?

Appendix

Table 4: Example instantiations in TemplateGSM.

Figure 7: Comparison of accuracy between the original LLMs ( Raw ) and the unedited models after REdit circuit reshaping ( REdit (w/o Edit) ) across three difficulty levels of logical reasoning tasks. REdit consistently provides modest gains, with the most notable improvements at Level 1.

Figure 8: Case study of circuits from reasoning patterns I and II before and after REdit circuit reshaping. REdit enhances intra-pattern consistency while eliminating inter-pattern overlap.

Figure 9: Trade-off of REdit

Dataset	Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	REdit
Level 1	Generality	$47.1~{\pm~0.1}$	$68.3~{\pm~0.1}$	$64.8~{\pm~0.4}$	$57.3~{\pm~0.1}$	$69.4~{\pm~0.4}$	$\textbf{70.8}~{\pm~0.1}$
Level 1	Locality	N/A	$72.2~{\pm~0.3}$	$76.3~{\pm~0.6}$	$76.5~{\pm~0.1}$	$76.3~{\pm~2.1}$	$\textbf{80.6}~{\pm~0.5}$
Level 2	Generality	$46.9~{\pm~0.3}$	$60.4~{\pm~0.1}$	$55.4~{\pm~0.4}$	$56.3~{\pm~0.4}$	$56.0~{\pm~0.1}$	$\textbf{69.1}~{\pm~0.1}$
Level 2	Locality	N/A	$73.8~{\pm~0.4}$	$73.7~{\pm~0.1}$	$\textbf{76.5}~{\pm~1.1}$	$73.4~{\pm~0.1}$	$78.2~{\pm~0.1}$
Level 3	Generality	$55.3~{\pm~0.1}$	$62.6~{\pm~0.1}$	$62.0~{\pm~0.4}$	$59.5~{\pm~0.4}$	$62.3~{\pm~0.5}$	$\textbf{65.6}~{\pm~0.2}$
Level 3	Locality	N/A	$87.6~{\pm~0.1}$	$87.0~{\pm~0.4}$	$\textbf{88.0}~{\pm~0.1}$	$87.3~{\pm~0.2}$	$87.3~{\pm~0.1}$

Appendix

Table 5: Results on ContextHub with Gemma-3-1B-IT as the backbone. The best scores are highlighted in bold .

Date Template	Instantiated Question
Today is Christmas Eve of [YEAR] . What is the date tomorrow in MM/DD/YYYY? A. [CHOICE_A] B. [CHOICE_B] C. [CHOICE_C] D. [CHOICE_D] E. [CHOICE_E] F. [CHOICE_F]	Today is Christmas Eve of 1909. What is the date tomorrow in MM/DD/YYYY? A. 10/25/1909 B. 12/26/1909 C. 12/28/1909 D. 12/25/1910 E. 12/27/1909 F. 12/25/1909

Appendix

Table 6: Example template and instantiated question from the Date dataset.

Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	REdit
Generality	$41.5~{\pm~1.1}$	$55.3~{\pm~1.0}$	$44.2~{\pm~0.2}$	$49.5~{\pm~0.7}$	$50.8~{\pm~0.4}$	$\textbf{57.6}~{\pm~0.3}$
Locality	N/A	$67.8~{\pm~0.3}$	$87.6~{\pm~0.8}$	$\textbf{91.2}~{\pm~1.1}$	$89.2~{\pm~1.1}$	$90.7~{\pm~0.5}$

Appendix

Table 7: Results on the Date dataset with Qwen-2.5-3B . The best scores are highlighted in bold .

Language Model Editing

Language model editing changes a model’s knowledge or behavior, or revises its outputs, while aiming to preserve unrelated capabilities.

See all 77 papers

Apr 7, 2026

Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models

Deploying Large Language Models (LLMs) in real-world dynamic environments raises the challenge of updating their pre-trained knowledge. While existing knowledge editing methods can reliably patch isolated facts, they frequently suffer from a reasoning gap, where the model recalls the edited fact but fails to utilize it in multi-step reasoning chains. Our analysis shows that multi-hop reasoning relies on sparse circuitry beyond direct factual recall, with functionally differentiated components contributing to intermediate processing and final-answer prediction. To bridge this gap, we introduce MCircKE (Mechanistic Circuit-based Knowledge Editing), a mechanistically grounded framework that enables a map-and-adapt editing procedure. MCircKE first identifies the causal circuits responsible for a specific reasoning task, capturing both the storage of the fact and the routing of its logical consequences. It then surgically update parameters exclusively within this mapped circuit. Extensive experiments on the MQuAKE-series benchmarks demonstrate the effectiveness of the proposed method for multi-hop reasoning in knowledge editing.

Tianyi Zhao, Yinhan He, Wendy Zheng +1

University of Virginia

Knowledge Editing Factual Knowledge in Language Models Multi-Hop Reasoning Circuits in Language Models

Ask PDF

Jul 15, 2026

Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models

The emergence of Chain-of-Thought (CoT) reasoning has significantly enhanced the ability of large language models (LLMs) to tackle complex, multi-step tasks. However, when errors occur, current interaction approaches typically involve re-generating another response that may make mistakes again, or users laboriously flag the faulty step in follow-up turns that may get responses <You are right, I made a mistake here> followed by similar errors recurring. To address this issue, we propose an efficient human intervention mechanism for precisely correcting reasoning errors in LLMs, termed Deep Interaction. Our approach enables direct editing of the original response, allowing erroneous parts to be corrected while preserving accurate reasoning steps. We refine the edited CoT into a distilled prompt, which then steers the LLM along the corrected reasoning path. Experimental results show that our method achieves over a 25% improvement in correction success rate and reduces token usage by approximately 40% on STEM tasks reasoning compared to baseline approaches.

Hefeng Zhou, Jinxuan Zhang, Jiong Lou +4

Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University

LLM Self-Correction LLM Prompting Human-in-the-Loop AI Large Reasoning Models

Ask PDF

Jun 26, 2026

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories

Current approaches to enhance Large Language Model (LLM) reasoning, such as Chain-of-Thought and "Wait" prompts, primarily encourage models to think more, yet often fail to guide them toward Truth. While Representation Editing (RepE) offers a intrinsic control, its application to dynamic reasoning trajectories remains underexplored. In this work, we bridge this gap by investigating the geometry of truth within unfolding reasoning chains. We uncover three critical insights: (1) Truth is encoded at the sentence level and is entangled with latent reasoning patterns; (2) Effective intervention follows an Uncertainty Principle and a Decay Effect, requiring localization to early, high-entropy forks; (3) Naive steering vectors suffer from noise, risking collateral damage to correct trajectories. Based on these findings, we propose DynaSteer, a dynamic RepE framework. DynaSteer employs pattern clustering to disentangle reasoning manifolds and utilizes Fisher-LDA to project purified truth. By dynamically monitoring lookahead entropy, it selectively steers and rolls back trajectories only when necessary. Comprehensive experimental results on several MATH benchmark verify the effectiveness of DynaSteer, and experiments on out-of-domain coding tasks further confirm its generalization ability. Our code is publicly available at https://github.com/tianlwang/DynaSteer.

Tianlong Wang, Yuhang Wang, Weibin Liao +5

National Engineering Research Center of Software Engineering, Peking University · School of Computer Science, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education +1

Language Model Steering Representation Geometry in Language Models LLM Reasoning

Ask PDF

Abstract

Language Model Editing

Explore similar work

Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models

Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories