Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping

Zhenyu Lei; Qiong Wu; Jianxiong Dong; Yinhan He; Emily Dodwell; Yushun Dong; Jundong Li

cs.CLMar 6, 2026

Reforming the Mechanism: Editing Reasoning Patterns in LLMs with Circuit Reshaping

Authors: Zhenyu Lei, Qiong Wu, Jianxiong Dong, Yinhan He, Emily Dodwell, Yushun Dong, Jundong Li

Organizations: University of Virginia · AT&T Chief Data Office · Florida State University

Abstract

Large language models (LLMs) often exhibit flawed reasoning ability that undermines reliability. Existing approaches to improving reasoning typically treat it as a general and monolithic skill, applying broad training which is inefficient and unable to target specific reasoning errors. We introduce Reasoning Editing, a paradigm for selectively modifying specific reasoning patterns in LLMs while preserving other reasoning pathways. This task presents a fundamental trade-off between Generality, the ability of an edit to generalize across different tasks sharing the same reasoning pattern, and Locality, the ability to preserve other reasoning capabilities. Through systematic investigation, we uncover the Circuit-Interference Law: Edit interference between reasoning patterns is proportional to the overlap of their neural circuits. Guided by this principle, we propose REdit, the first framework to actively reshape neural circuits before editing, thereby modulating interference between reasoning patterns and mitigating the trade-off. REdit integrates three components: (i) Contrastive Circuit Reshaping, which directly addresses the generality-locality trade-off by disentangling overlapping circuits; (ii) Meta-Contrastive Learning, which extends transferability to novel reasoning patterns; and (iii) Dual-Level Protection, which preserves preexisting abilities by constraining reshaping update directions and regularizing task-level predictions. Extensive experiments with Qwen-2.5-3B on propositional logic reasoning tasks across three difficulty levels demonstrate that REdit consistently achieves superior generality and locality compared to baselines, with additional validation in mathematics showing broader potential. Our code is available at https://github.com/LzyFischer/REdit.

Figures & tables

Panel of Figure 1 — Figure 1: LLM reasoning deficiencies and editing trade-off.

Figure 2: Correlation between circuit distance and interference. (a–c) Scatter plots with regression lines show that larger distances consistently correspond to reduced interference across different distance metrics. (d) Density plots of Pearson correlations confirm consistent negative associations.

Figure 3: Editing success rates across methods on ContextHub. REdit achieves success rates comparable to other approaches, confirming that it does not compromise the model’s fundamental editing capabilities.

Dataset	Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	Ours
Level 1	Generality	$60.7~{\pm~2.3}$	$\underline{72.2}~{\pm 1.4}$	$63.8~{\pm~2.9}$	$67.8~{\pm 3.2}$	$67.9~{\pm 1.9}$	$\textbf{74.1}~{\pm~1.6}$
Level 1	Locality	N/A	$61.5~{\pm 0.7}$	$84.9~{\pm~1.6}$	$\underline{89.8}~{\pm 3.1}$	$87.0~{\pm 0.9}$	$\textbf{94.3}~{\pm~0.4}$
Level 2	Generality	$53.2~{\pm~1.4}$	$\underline{63.6}~{\pm 2.9}$	$58.4~{\pm~0.1}$	$61.3~{\pm 1.1}$	$58.8~{\pm 1.5}$	$\textbf{64.8}~{\pm~1.2}$
Level 2	Locality	N/A	$59.4~{\pm 4.1}$	$91.5~{\pm~0.0}$	$93.1~{\pm 0.1}$	$\underline{93.3}~{\pm 0.0}$	$\textbf{94.3}~{\pm~0.5}$
Level 3	Generality	$45.1~{\pm~1.6}$	$52.6~{\pm 0.4}$	$50.1~{\pm~0.8}$	$51.5~{\pm 3.3}$	$\underline{54.2}~{\pm 0.8}$	$\textbf{55.0}~{\pm~1.6}$
Level 3	Locality	N/A	$52.3~{\pm 1.0}$	$92.3~{\pm~2.8}$	$\textbf{94.6}~{\pm 2.7}$	$92.2~{\pm 0.7}$	$\underline{94.4}~{\pm~0.8}$

Table 1: Main results on ContextHub evaluated with generality and locality metrics. The best and second-best scores are highlighted in bold and underlined , respectively. Raw denotes the performance of the unedited LLM. For BIMT, we apply the same LoRA-based editing method as in REdit.

Dataset	Metric	Raw	w/o MCL	w/o NSP	w/o PDP	Ours
Level 1	Generality	$60.7~{\pm~2.3}$	${72.9}~{\pm~0.4}$	$73.3~{\pm~0.2}$	$\underline{73.4}~{\pm~0.5}$	$\textbf{74.1}~{\pm~1.6}$
Level 1	Locality	N/A	$\underline{90.7}~{\pm~1.8}$	$89.5~{\pm~0.3}$	$90.1~{\pm~2.5}$	$\textbf{94.3}~{\pm~0.4}$
Level 2	Generality	$53.2~{\pm~1.4}$	$\underline{62.5}~{\pm~0.3}$	$62.4~{\pm~1.6}$	$61.3~{\pm~2.0}$	$\textbf{64.8}~{\pm~1.2}$
Level 2	Locality	N/A	$\textbf{94.9}~{\pm~0.6}$	$93.0~{\pm~1.8}$	$94.0~{\pm~0.8}$	$\underline{94.3}~{\pm~0.5}$
Level 3	Generality	$45.1~{\pm~1.6}$	$\underline{53.8}~{\pm~1.3}$	$50.9~{\pm~0.6}$	$51.8~{\pm~0.6}$	$\textbf{55.0}~{\pm~1.6}$
Level 3	Locality	N/A	$\underline{93.7}~{\pm~1.3}$	$92.8~{\pm~1.1}$	$92.8~{\pm~1.2}$	$\textbf{94.4}~{\pm~0.8}$

Table 2: Ablation studies on ContextHub evaluated with generality and locality metrics. The best and second-best scores are highlighted in bold and underlined , respectively.

Figure 4: Performance on unseen reasoning patterns after circuit reshaping with different ratios for training. REdit consistently outperforms baselines without reshaping.

Panel of Figure 5 — Figure 5: Circuit–interference relationship before and after circuit reshaping. (a,b) Scatter plots of intra- and inter-pattern measurements show improved separability in interference and circuit distance. (c) Silhouette scores across reasoning patterns indicate consistent gains in cluster separation.

Figure 6: Evaluation on mathematical reasoning benchmark TemplateGSM.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Abstract Instance	Contextual Instance
$(\texttt{vxkgr}\lor\texttt{caunc})\to\texttt{ybyz}$ . Given ybyz is False, what is the value of caunc ?	If an area of land has experienced significant uplift or been shaped by powerful erosional forces, then the terrain will feature tall, steep mountains. Given that the area does not have tall, steep mountains, can it be determined if powerful erosional forces have shaped the land?

Appendix

Table 3: Level-1 example instantiations in ContextHub.

Math Template	Instantiated Problem
[NAME] sold [NUM1] [ITEM] to [her/his/their] friends in April at a [LOCATION] in [COUNTY] , [STATE] . In May, [PRONOUN] sold [NUM2] [ITEM] . How many [ITEM] did [NAME] sell altogether in April and May?	Rosy Plascencia sold 238 air fryers to her friends in April at a yoga studio boutique in Bracken County, Kentucky. In May, they sold 119 air fryers. How many air fryers did Rosy Plascencia sell altogether in April and May?

Appendix

Table 4: Example instantiations in TemplateGSM.

Figure 7: Comparison of accuracy between the original LLMs ( Raw ) and the unedited models after REdit circuit reshaping ( REdit (w/o Edit) ) across three difficulty levels of logical reasoning tasks. REdit consistently provides modest gains, with the most notable improvements at Level 1.

Figure 8: Case study of circuits from reasoning patterns I and II before and after REdit circuit reshaping. REdit enhances intra-pattern consistency while eliminating inter-pattern overlap.

Figure 9: Trade-off of REdit

Dataset	Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	REdit
Level 1	Generality	$47.1~{\pm~0.1}$	$68.3~{\pm~0.1}$	$64.8~{\pm~0.4}$	$57.3~{\pm~0.1}$	$69.4~{\pm~0.4}$	$\textbf{70.8}~{\pm~0.1}$
Level 1	Locality	N/A	$72.2~{\pm~0.3}$	$76.3~{\pm~0.6}$	$76.5~{\pm~0.1}$	$76.3~{\pm~2.1}$	$\textbf{80.6}~{\pm~0.5}$
Level 2	Generality	$46.9~{\pm~0.3}$	$60.4~{\pm~0.1}$	$55.4~{\pm~0.4}$	$56.3~{\pm~0.4}$	$56.0~{\pm~0.1}$	$\textbf{69.1}~{\pm~0.1}$
Level 2	Locality	N/A	$73.8~{\pm~0.4}$	$73.7~{\pm~0.1}$	$\textbf{76.5}~{\pm~1.1}$	$73.4~{\pm~0.1}$	$78.2~{\pm~0.1}$
Level 3	Generality	$55.3~{\pm~0.1}$	$62.6~{\pm~0.1}$	$62.0~{\pm~0.4}$	$59.5~{\pm~0.4}$	$62.3~{\pm~0.5}$	$\textbf{65.6}~{\pm~0.2}$
Level 3	Locality	N/A	$87.6~{\pm~0.1}$	$87.0~{\pm~0.4}$	$\textbf{88.0}~{\pm~0.1}$	$87.3~{\pm~0.2}$	$87.3~{\pm~0.1}$

Appendix

Table 5: Results on ContextHub with Gemma-3-1B-IT as the backbone. The best scores are highlighted in bold .

Date Template	Instantiated Question
Today is Christmas Eve of [YEAR] . What is the date tomorrow in MM/DD/YYYY? A. [CHOICE_A] B. [CHOICE_B] C. [CHOICE_C] D. [CHOICE_D] E. [CHOICE_E] F. [CHOICE_F]	Today is Christmas Eve of 1909. What is the date tomorrow in MM/DD/YYYY? A. 10/25/1909 B. 12/26/1909 C. 12/28/1909 D. 12/25/1910 E. 12/27/1909 F. 12/25/1909

Appendix

Table 6: Example template and instantiated question from the Date dataset.

Metric	Raw	BIMT	LoRA	ROME	AlphaEdit	REdit
Generality	$41.5~{\pm~1.1}$	$55.3~{\pm~1.0}$	$44.2~{\pm~0.2}$	$49.5~{\pm~0.7}$	$50.8~{\pm~0.4}$	$\textbf{57.6}~{\pm~0.3}$
Locality	N/A	$67.8~{\pm~0.3}$	$87.6~{\pm~0.8}$	$\textbf{91.2}~{\pm~1.1}$	$89.2~{\pm~1.1}$	$90.7~{\pm~0.5}$

Appendix

Table 7: Results on the Date dataset with Qwen-2.5-3B . The best scores are highlighted in bold .

Language Model Editing

Language model editing changes a model’s knowledge or behavior, or revises its outputs, while aiming to preserve unrelated capabilities.

See all 77 papers

Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models
Apr 7, 2026Tianyi Zhao, Yinhan He, Wendy Zheng +1Knowledge Editing Factual Knowledge in Language Models
University of Virginia
Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models
Jul 15, 2026Hefeng Zhou, Jinxuan Zhang, Jiong Lou +4LLM Self-Correction LLM Prompting
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University
Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories
Jun 26, 2026Tianlong Wang, Yuhang Wang, Weibin Liao +5Language Model Steering Representation Geometry in Language Models
National Engineering Research Center of Software Engineering, Peking University · School of Computer Science, Peking University · Key Laboratory of High Confidence Software Technologies, Ministry of Education +1

Apr 7, 2026J/K move · Enter open · S save

Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models

Tianyi Zhao, Yinhan He, Wendy Zheng, Chen Chen

University of Virginia

Knowledge Editing Factual Knowledge in Language Models Multi-Hop Reasoning Circuits in Language Models

Open paper AskPDF arXiv

Deploying Large Language Models (LLMs) in real-world dynamic environments raises the challenge of updating their pre-trained knowledge. While existing knowledge editing methods can reliably patch isolated facts, they frequently suffer from a reasoning gap, where the model recalls the edited fact but fails to utilize it in multi-step reasoning chains. Our analysis shows that multi-hop reasoning relies on sparse circuitry beyond direct factual recall, with functionally differentiated components contributing to intermediate processing and final-answer prediction. To bridge this gap, we introduce MCircKE (Mechanistic Circuit-based Knowledge Editing), a mechanistically grounded framework that enables a map-and-adapt editing procedure. MCircKE first identifies the causal circuits responsible for a specific reasoning task, capturing both the storage of the fact and the routing of its logical consequences. It then surgically update parameters exclusively within this mapped circuit. Extensive experiments on the MQuAKE-series benchmarks demonstrate the effectiveness of the proposed method for multi-hop reasoning in knowledge editing.

Abstract

Language Model Editing

Explore similar work

Addressing the Reasoning Gap: Mechanistic Circuit-Based Knowledge Editing in Large Language Models

Deep Interaction: An Efficient Human-AI Interaction Method for Large Reasoning Models

Search for Truth from Reasoning: A Dynamic Representation Editing Framework for Steering LLM Trajectories