CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing

Manit Baser; Alperen Yildiz; Dinil Mon Divakaran; Mohan Gurusamy

cs.LGMar 11, 2026

CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing

Authors: Manit Baser, Alperen Yildiz, Dinil Mon Divakaran, Mohan Gurusamy

Organizations: National University of Singapore, Singapore · A*STAR Institute for Infocomm Research (A*STAR I2R), Singapore

Abstract

The static knowledge representations of large language models (LLMs) inevitably become outdated or incorrect over time. While model-editing techniques offer a promising solution by modifying a model's factual associations, they often produce unpredictable ripple effects, which are unintended behavioral changes that propagate even to the hidden space. In this work, we introduce CLaRE, a lightweight representation-level technique to identify where these ripple effects may occur. Unlike prior gradient-based methods, CLaRE quantifies entanglement between facts using forward activations from a single intermediate layer, avoiding costly backward passes. To enable systematic study, we prepare and analyse a corpus of 11,427 facts drawn from three existing datasets. Using CLaRE, we compute large-scale entanglement graphs of this corpus for multiple models, capturing how local edits propagate through representational space. These graphs enable stronger preservation sets for model editing, audit trails, efficient red-teaming, and scalable post-edit evaluation. In comparison to baselines, CLaRE achieves an average of 62.2% improvement in Spearman correlation with ripple effects while being $2.74\times$ faster, and using $2.85\times$ less peak GPU memory. Besides, CLaRE requires only a fraction of the storage needed by the baselines to compute and preserve fact representations. Our entanglement graphs and corpus are available at https://github.com/manitbaser/CLaRE.

Figures & tables

Figure 1: A targeted update to a political fact may inadvertently alter the model’s prediction for an unrelated musical fact, despite no semantic connection. This demonstrates how edits can trigger ripple effects far beyond the intended factual neighborhood.

Panel of Figure 2 — Figure 2: For each fact, GradSim computes the entire gradient, while CLaRE uses a single forward pass up till the last critical layer, enabling faster and scalable entanglement mapping.

Panel of Figure 3 — Figure 3: Correlation patterns for AlphaEdit: entanglement vs. $\ell_{2}$ logit shift (left) and $|\Delta\log P(y)|$ (right).

Model	Metric ( $\rho_{s}$ )	Method	Editing Techniques
Model	Metric ( $\rho_{s}$ )	Method	MEMIT	RECT	ROME	PRUNE	AlphaEdit
GPT-J	$\|\Delta\log P(y)\|$	GradSim	0.582	0.549	0.509	0.597	0.547
	$\|\Delta\log P(y)\|$	CLaRE	0.838 (+44.0%)	0.805 (+46.6%)	0.797 (+56.6%)	0.854 (+43.0%)	0.805 (+47.2%)
	$\ell_{2}$ logit shift	GradSim	0.677	0.646	0.634	0.691	0.663
	$\ell_{2}$ logit shift	CLaRE	0.913 (+34.9%)	0.878 (+35.9%)	0.857 (+35.2%)	0.917 (+32.7%)	0.875 (+32.0%)
GPT2-XL	$\|\Delta\log P(y)\|$	GradSim	0.545	0.531	0.544	0.558	0.550

Table 1: Spearman Correlation Scores for CLaRE and GradSim across

|\Delta\log P(y)|

and

\ell_{2}

logit shift.

Panel of Figure 4 — Figure 4: Performance comparison between CLaRE and GradSim in terms of Spearman correlation ( $\rho_{s}$ ). The left panel shows $\rho_{s}$ between entanglement values and $\ell_{2}$ logit shift, and right panel shows $\rho_{s}$ between entanglement values and $|\Delta\log P(y)|$ . CLaRE (wider, transparent bars) consistently achieves higher $\rho_{s}$ than GradSim (narrower, solid bars).

Method	GPT2-XL	Qwen2.5	Qwen2	Llama3	Llama3.2	GPT-J	Mistral
Total Processing Time (seconds)
GradSim	0.796	0.915	0.868	0.917	0.774	0.763	0.833
CLaRE	0.305	0.305	0.307	0.305	0.307	0.305	0.306
Speed-up	2.610×	3.000×	2.827×	3.007×	2.521×	2.502×	2.722×
Size of representation used for computation (bytes)
GradSim	6.23e9	3.05e10	3.05e10	3.21e10	1.29e10	2.42e10	2.90e10

Table 2: Performance comparison between CLaRE and GradSim. Lower values for all quantities indicate better performance. The ratio rows (e.g., Speed-up) show how many times CLaRE is faster or more memory-efficient than GradSim. For example, a speed-up of 3.0× means CLaRE completes the task 3.0 times faster than GradSim.

Panel of Figure 5 — Figure 5: Computational efficiency comparison. Closer to center => better performance.

Model	Layer Type	AlphaEdit	MEMIT	PRUNE	RECT	ROME
Llama3	$\mathcal{L}=9$	65.16%	63.76%	64.35%	61.73%	70.54%
	Maximum $\rho_{s}$ (any layer)	65.16%	64.16%	64.84%	61.73%	70.54%
	Difference from peak (p.p.)	0.00	0.40	0.49	0.00	0.00
GPT-J	$\mathcal{L}=9$	89.30%	80.13%	92.72%	90.26%	88.46%
	Maximum $\rho_{s}$ (any layer)	89.72%	80.13%	92.97%	90.26%	88.58%
	Difference from peak (p.p.)	0.42	0.00	0.25	0.00	0.12

Table 3:

\rho_{s}

between CLaRE and ripple effect (

\ell_{2}

logit shift) in control facts observed at

\mathcal{L}

, compared to maximum value possible at any layer. "Difference from peak" reports the absolute deviation in percentage points (p.p.).

Figure 7: An entanglement cluster in GPT-J with its ten most entangled subjects listed with their number and percentage of facts. It consists of 1,149 facts and 397 subjects, with 87.9% of edges between inter-subject facts. Node color indicates subject identity.

Appendix figures & tables35 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: Layerwise Spearman correlation ( $\rho_{s}$ ) between CLaRE and observed ripple magnitudes. Correlation peaks around the last critical layer indicate that it is most informative about entanglement estimation.

Component	GradSim	CLaRE
Forward pass (per fact)	$\mathcal{O}(L\cdot d^{2}+N^{2}\cdot L\cdot d)$	$\mathcal{O}(L\cdot d^{2}+N^{2}\cdot L\cdot d)$
Loss computation (per fact)	$\mathcal{O}(d\cdot v)$	—
Backward pass (per fact)	$\mathcal{O}(L\cdot d^{2})$	—
Cosine similarity computation	$\mathcal{O}(L\cdot d^{2})$	$\mathcal{O}(d)$
Total Time Complexity	$\sim\mathcal{O}(L\cdot d^{2})$	$\sim\mathcal{O}(L\cdot d^{2})$
Gradient storage (per fact)	$\mathcal{O}(L\cdot d^{2})$	—

Appendix

Table 4: Step-by-step comparison of time and space complexity between GradSim and CLaRE for computing entanglement between two facts.

Model	Method 1: CLaRE			Method 2: GradSim
Model	Forward ( $\times 2$ Vec)	Sim.	Total	Forward ( $\times 2$ Grad)	Loss ( $\times 2$ Grad)	Backward ( $\times 2$ Grad)	Sim.	Total
GPT-J	0.204	0.101	0.305	0.251	0.205	0.205	0.102	0.763
GPT2-XL	0.204	0.101	0.305	0.284	0.205	0.205	0.102	0.796
Llama3.2	0.206	0.101	0.307	0.262	0.204	0.205	0.102	0.773
Llama3	0.204	0.101	0.305	0.305	0.205	0.205	0.202	0.917
Mistral	0.205	0.101	0.306	0.304	0.205	0.223	0.101	0.833

Appendix

Table 5: Detailed time breakdown (seconds) for CLaRE and GradSim. CLaRE follows: Forward Pass (

\times 2

)

\oplus

Cosine similarity (Sim.) of two vectors (Vec). GradSim follows: Forward (

\times 2

)

\oplus

Loss (

\times 2

)

\oplus

Backward (

\times 2

)

\oplus

Cosine similarity of gradients (Grad).

Model	Method 1: CLaRE			Method 2: GradSim
	Forward Pass		Sim	Gradient 1 Steps			Gradient 2 Steps			Sim
	Fwd 1	Fwd 2	Sim	Fwd	Loss	Bwd	Fwd	Loss	Bwd	Sim
GPT-J	23.815	23.905	22.588	33.551	33.550	45.167	56.134	56.133	67.710	68.474
GPT2-XL	6.968	7.016	6.013	9.032	9.029	12.275	15.003	15.000	18.210	18.212
Llama3.2	12.940	12.995	12.001	18.042	18.037	25.475	30.048	30.043	37.439	37.445
Llama3	31.459	31.539	29.946	44.012	44.007	59.898	73.967	73.961	89.813	91.765

Appendix

Table 6: Step-by-step peak GPU memory consumption (GB). CLaRE steps: Fwd (forward pass) 1

\rightarrow

Fwd 2

\rightarrow

Sim (Cosine similarity of two vectors). GradSim steps: Gradient 1 (Fwd

\rightarrow

Loss calculation

\rightarrow

Bwd (backward pass)

\rightarrow

Gradient 2 (Fwd

\rightarrow

Loss

\rightarrow

Bwd)

\rightarrow

Sim (Cosine similarity of two gradients).

Figure 9: Distribution of cosine similarities between CLaRE representations across 11,427 facts.

Model	Index of $\mathcal{L}$ / Number of layers
Baichuan-7B	0.28125
GPT-J-6B	0.32143
GPT-2-XL	0.37500
InternLM-7B	0.28125
Llama-7B	0.28125
Llama3.2-3B	0.34615

Appendix

Table 7: Ratio of the last critical layer to total number of layers for various models, taken from the EasyEdit repository. Values cluster around one-third depth, supporting the use of a one-third-depth heuristic when the critical layer is unknown.

Model	Layer Type	AlphaEdit	MEMIT	PRUNE	RECT	ROME
Llama-3	1/3-depth layer = 11	64.31%	62.74%	63.41%	61.02%	69.53%
	Maximum $\rho_{s}$ (any layer)	65.16%	64.16%	64.84%	61.73%	70.54%
	Difference from peak (p.p.)	0.85	1.42	1.43	0.71	1.01
GPT-J	1/3-depth layer = 9	89.30%	80.13%	92.72%	90.26%	88.46%
	Maximum $\rho_{s}$ (any layer)	89.72%	80.13%	92.97%	90.26%	88.58%
	Difference from peak (p.p.)	0.42	0.00	0.25	0.00	0.12

Appendix

Table 8: Spearman correlation (

\rho_{s}

) between CLaRE and ripple effect (

\ell_{2}

logit shift) in control facts at the one-third-depth layer (indexed by layer number), compared with the maximum value observed across all layers for each editing method. "Difference from peak" reports the absolute deviation in percentage points (p.p.).

Metric	Evaluation Focus	Temporal Application
CLaRE	Measures shared subspaces between facts using cosine similarity at the last critical layer.	Pre-edit
Efficacy	Measures whether the model correctly outputs the new target fact after editing.	Post-edit
Locality	Measures whether the model’s predictions for unrelated facts remain unchanged.	Post-edit
Generalization	Measures whether the edit holds across different semantic phrasings of the same fact.	Post-edit
$\|\Delta\log P(y)\|$	Measures the change in log-probability of the correct answer before and after editing.	Post-edit
$\ell_{2}$ logit shift	Measures the global geometric displacement in the output logit distribution.	Post-edit

Appendix

Table 9: Comparison between CLaRE , ripple-effect measurements, and standard editing evaluation metrics.

Model	Edited Fact	Entangled Fact	CLaRE	$\|\Delta\log P(y)\|$
Llama3	The official language of Germany is (German $\rightarrow$ English)	The official language of Poland is: Polish	0.84	1.43
Llama3	Queen’s Park F.C. is associated with the sport of (association football $\rightarrow$ basketball)	St Patrick’s Athletic F.C. is headquartered in: Dublin	0.87	1.17
GPT2-XL	The official language of Italian Republic is (Italian $\rightarrow$ French)	The official language of Dutch Republic is: Dutch	0.93	2.12
GPT2-XL	The capital of Denmark is (Copenhagen $\rightarrow$ London)	The capital of Sweden is: Stockholm	0.88	0.07
GPT-J	The name of the capital city of France is (Paris $\rightarrow$ London)	The name of the capital city of Russia is: Moscow	0.87	1.03
GPT-J	The name of the award Norman Mailer won is (Pulitzer Prize for Fiction $\rightarrow$ Academy Award)	Ernest Hemingway is a citizen of: United States of America	0.79	1.16

Appendix

Table 10: Representative examples where fact pairs exhibit high CLaRE similarity but minimal observed ripple effects. These cases illustrate that CLaRE measures susceptibility rather than guaranteeing interference.

Panel of Figure 10 — Figure 10: Correlation patterns for RECT across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 11 — Figure 11: Correlation patterns for RECT across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 12 — Figure 12: Correlation patterns for MEMIT across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 13 — Figure 13: Correlation patterns for MEMIT across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 14 — Figure 14: Correlation patterns for ROME across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 15 — Figure 15: Correlation patterns for ROME across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 16 — Figure 16: Correlation patterns for PRUNE across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 17 — Figure 17: Correlation patterns for PRUNE across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 18 — Figure 18: CLaRE correlation patterns for AlphaEdit across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 19 — Figure 19: CLaRE correlation patterns for AlphaEdit across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 20 — Figure 20: CLaRE correlation patterns for RECT across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 21 — Figure 21: CLaRE correlation patterns for RECT across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 22 — Figure 22: CLaRE correlation patterns for MEMIT across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 23 — Figure 23: Correlation patterns for MEMIT across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 24 — Figure 24: CLaRE correlation patterns for ROME across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 25 — Figure 25: CLaRE correlation patterns for ROME across different models for entanglement vs $|\Delta\log P(y)|$ .

Panel of Figure 26 — Figure 26: CLaRE correlation patterns for PRUNE across different models for entanglement vs $\ell_{2}$ logit shift.

Panel of Figure 27 — Figure 27: CLaRE correlation patterns for PRUNE across different models for entanglement vs $|\Delta\log P(y)|$ .

Phase	Complexity
Clean run	$O(N\cdot T\cdot L\cdot(T\cdot H+H\cdot D))$
Corrupted run	$O(N\cdot T\cdot L\cdot(T\cdot H+H\cdot D))$
Restoration passes	$O(N\cdot T^{2}\cdot L^{2}\cdot(T\cdot H+H\cdot D))$
Total	$\mathbf{O(N\cdot T^{2}\cdot L^{2}\cdot(T\cdot H+H\cdot D))}$

Appendix

Table 11: Phase-wise complexity analysis of causal tracing for critical layer identification.

Panel of Figure 32 — Figure 32: EleutherAI GPT-J-6B: Clusters 01-02

Panel of Figure 33 — Figure 33: EleutherAI GPT-J-6B: Clusters 03-08

Panel of Figure 34 — Figure 34: EleutherAI GPT-J-6B: Clusters 09-13

Panel of Figure 35 — Figure 35: GPT2-XL: Clusters 01-06

Figure 36: GPT2-XL: Cluster 07 54 facts | 9 subjects | 29.4% cross-subject edges

Panel of Figure 37 — Figure 37: Llama3-8B: Clusters 01-04

Panel of Figure 38 — Figure 38: Llama3-8B: Clusters 05-07

Explore similar work

Feb 17, 2026

CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing

A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.

Zarif Ikram, Arad Firouzkouhi, Stephen Tu +2

University of Southern California

Language Model Editing

Ask PDF

Mar 16, 2026

SCAN: Sparse Circuit Anchor Interpretable Neuron for Lifelong Knowledge Editing

Large Language Models (LLMs) often suffer from catastrophic forgetting and collapse during sequential knowledge editing. This vulnerability stems from the prevailing dense editing paradigm, which treats models as black boxes and relies on coarse-grained parameter interventions that inevitably disrupt preserved knowledge. To address this, we propose SCAN (a sparse editing framework based on Sparse Circuit Anchored Neuron) which transforms editing into a mechanism-aware manipulation by constructing a knowledge circuit via Sparse Transcoders. Experiments on Gemma2, Qwen3, and Llama3.1 across CounterFact, ZsRE and WikiFactDiff demonstrate that SCAN achieves a superior performance, maintaining model integrity on benchmarks like MMLU and GSM8K even after 3,000 sequential edits, whereas other existing methods deteriorate progressively as editing accumulates, eventually resulting in model collapse.

Yuhuan Liu, Haitian Zhong, Xinyuan Xia +3

New Laboratory of Pattern Recognition (NLPR), State Key Laboratory of Multimodal Artificial Intelligence Systems (MAIS), Institute of Automation, Chinese Academy of Sciences · Cuiying Honors College, Lanzhou University · Zhongguancun Academy +1

Knowledge Editing Language Model Editing Catastrophic Forgetting Continual Learning for LLMs

Ask PDF

Jun 21, 2026

Orthogonal Representation Editing: Decoupling Semantic Entanglement in Batch Knowledge Editing of LLMs

Knowledge editing aims to efficiently update factual information in Large Language Models (LLMs) without full retraining. However, existing methods still suffer from performance degradation in batch knowledge editing. We identify that semantic representation entanglement, such as overlapping concepts and shared syntactic patterns, accumulates interference in the representation space and reduces editing precision. To bridge this gap, in this paper, we propose Orthogonal Representation Editing (ORE), which performs edits in the hidden representation space of LLMs by constructing a general semantic subspace and enforcing orthogonal constraints on edit vectors, effectively decoupling semantic entanglement. Furthermore, we introduce a gated non-linear representation head to enable adaptive learning of editing locations and precise control over knowledge injection. Extensive experiments show that ORE outperforms existing methods and achieves superior performance in cross-lingual knowledge editing scenarios. We release our code at https://github.com/YVVH/ORE.

Wenhao Yu, Zhicong Lu, Bo Lv +4

School of Computer Science and Technology, Tianjin University · Kexin Technology · University of Chinese Academy of Sciences +2

Knowledge Editing Model Editing Language Model Editing Representation Geometry in Language Models

Ask PDF