CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing

Zarif Ikram; Arad Firouzkouhi; Stephen Tu; Mahdi Soltanolkotabi; Paria Rashidinejad

cs.LGFeb 17, 2026

CrispEdit: Low-Curvature Projections for Scalable Non-Destructive LLM Editing

Authors: Zarif Ikram, Arad Firouzkouhi, Stephen Tu, Mahdi Soltanolkotabi, Paria Rashidinejad

Organizations: University of Southern California

Abstract

A central challenge in large language model (LLM) editing is capability preservation: methods that successfully change targeted behavior can quietly game the editing proxy and corrupt general capabilities, producing degenerate behaviors reminiscent of proxy/reward hacking. We present CrispEdit, a scalable and principled second-order editing algorithm that treats capability preservation as an explicit constraint, unifying and generalizing several existing editing approaches. CrispEdit formulates editing as constrained optimization and enforces the constraint by projecting edit updates onto the low-curvature subspace of the capability-loss landscape. At the crux of CrispEdit is expressing capability constraint via Bregman divergence, whose quadratic form yields the Gauss-Newton Hessian exactly and even when the base model is not trained to convergence. We make this second-order procedure efficient at the LLM scale using Kronecker-factored approximate curvature (K-FAC) and a novel matrix-free projector that exploits Kronecker structure to avoid constructing massive projection matrices. Across standard model-editing benchmarks, CrispEdit achieves high edit success while keeping capability degradation below 1% on average across datasets, significantly improving over prior editors.

Figures & tables

Figure 1: Comparison overview of CrispEdit . CrispEdit achieves strong edit reliability and generality, with and without QA context, while preserving broad base capabilities (MMLU, GSM8K, IFEval, ARC-C, TruthfulQA) on LLaMA-3-8B-Instruct .

Figure 2: Geometric interpretation of CrispEdit compared to baseline editing strategies. Top left: Standard gradient descent effectively minimizes edit loss but moves perpendicular to the capability contours, resulting in high capability loss (degradation). Top right: Projecting onto the nullspace of activation covariance is overly conservative; it preserves representations but restricts the update too heavily to successfully optimize the edit loss. Bottom: CrispEdit projects the update onto the low-curvature subspace of the capability loss. This allows changes in representations to satisfy the edit while moving along the “valley” of the landscape to maintain general model capabilities.

Figure 3: Tradeoff between pre-training accuracy (capability preservation) and post-training performance (edit efficacy) for different nullspace projection methods. We fine-tune a LeNet-5 model pre-trained on MNIST on Fashion-MNIST in the

\gamma

-approximate nullspace of the embeddings (Adam-NSCL) Hessian along with Hessian approximations Gauss-Newton Hessian, K-FAC, and EK-FAC ( CrispEdit ), over a range of energy thresholds

\gamma

Data	Method	Edited Capabilities				Base Capabilities					Time
		QA Context		No Context
		Rel	Gen	Rel	Gen	MMLU	IFEval	TruthfulQA	ARC-C	GSM8K
ZsRE	LLaMA-3-8B-Instruct	2.1	1.7	2.9	2.1	69.5	69.3	50.7	58.0	73.5
	MEMIT	0.1	0.0	0.1	0.1	22.9	0.0	51.3	23.5	0.0	9h 27m
	AlphaEdit	70.1	60.6	48.1	39.4	52.7	47.7	46.3	40.5	45.5	7h 19m
	Adam-NSCL	16.6	15.5	1.9	2.0	69.2	29.6	50.8	42.0	39.5	29m 19s

Table 1: Comparison of CrispEdit with existing methods on editing LLaMA-3-8B-Instruct . Rel and Gen denote reliability and generalization. We edit 3,000 samples from three datasets, evaluate edits with WILD, and measure base capability on five benchmarks. Values that are best or within 5% of best are in bold.

Figure 4: Runtime comparison of CrispEdit with other methods. We apply a number of model editing methods to edit LLaMA-3-8B-Instruct on 3,000 ZsRE samples and measure the wall-clock time for execution.

Figure 5: Effect of capability dataset size

n

on editing performance and base capability preservation. We edit LLaMA-3-8B-Instruct on 3,000 ZsRE samples using CrispEdit for a range of

n

and measure the editing performance and base capability preservation.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 8 — Figure 8: The complete prompt used to employ an LLM as a judge. The judge provides binary assessments (correct or incorrect) based on a given question, gold target answer, and predicted answer.

Hyperparameter	Value
Editing layers ( LLaMA-3-8B-Instruct )	{19, 20, 21, 22, 23}
Editing layers ( Qwen-2.5-1.5B-Instruct )	{4, 5, 6}
Number of steps	25
Early stopping	0.01
Batch size	32
Chunk size ( CrispEdit - Seq )	100

Appendix

Table 2: Default hyperparameters used for CrispEdit and CrispEdit - Seq .

Model Editing Case Study 1
Editing Prompt	What voice type does Marina Rebeka have?
Edit Target	mezzo-srano
Generation Output
Adam-NSCL	mezzo-srano -srano-srano-srano-srano-srano-srano-srano-srano-srano-srano-srano-srano-srano-srano-srano
LocBFFT	mezzo-oprano
AlphaEdit	mezzo-soprano

Appendix

Table 10

Method	ZsRE			CounterFact			WikiBigEdit
Method	Rel	Gen	Spec	Rel	Gen	Spec	Rel	Gen	Spec
Llama 3 8B Instruct	25.7	25.1	37.8	0.9	1.2	89.4	34.0	34.8	32.8
MEMIT	0.0	0.0	0.0	0.0	0.0	49.4	0.5	0.5	0.0
AlphaEdit	86.7	77.8	32.4	94.3	72.0	69.1	95.0	89.0	42.0
Adam-NSCL	98.8	92.4	22.1	99.5	81.5	47.7	99.7	97.5	36.4
LocBF-FT	99.1	91.1	34.5	99.7	72.7	44.6	99.9	96.8	42.8

Appendix

Table 3: Comparison of CrispEdit with existing methods on editing LLaMA-3-8B-Instruct in the teacher-forcing evaluation pipeline. Rel , gen , Spec denote reliability, generality, and specificity, respectively. We perform model editing on 3,000 samples of three representative datasets and evaluate editing performance and base performance following teacher-forcing setup of Meng et al. (2023) ; Meng et al. (2022) ; Fang et al. (2025) . Results that are the highest or within 5% of the highest results are highlighted in bold.

Data	Method	Edited Capabilities				Base Capabilities
		QA Context		No Context
		Rel	Gen	Rel	Gen	MMLU	IFEval	TruthfulQA	ARC-C	GSM8K
ZsRE 10K	LLaMA-3-8B-Instruct	2.0	1.5	2.9	2.1	69.5	69.3	50.7	58.0	73.5
	LocBF-FT	53.5	47.7	11.5	11.6	68.0	67.6	50.7	50.0	73.0
	UltraEdit	20.1	16.7	12.6	10.4	67.9	68.9	49.8	46.0	73.0
	Adam-NSCL	1.2	1.1	0.4	0.7	68.2	14.8	54.0	35.0	2.0

Appendix

Table 4: Influence of scaling to larger editing dataset. Rel and Gen denote reliability and generality, respectively. We perform model editing on 10,000 samples of ZsRE and evaluate editing performance with WILD framework and base performance with five representative benchmarks. Results that are the highest or within 5% of the highest results are highlighted in bold.

Data	Model	Edited Capabilities				Base Capabilities
		QA Context		No Context
		Rel	Gen	Rel	Gen	MMLU	IFEval	TruthfulQA	ARC-C	GSM8K
ZsRE	Qwen 2.5 1.5B	3.5	4.0	2.3	2.0	61.9	48.3	50.9	52.0	58.0
	FT	35.4	29.6	32.2	25.5	50.0	24.8	49.8	34.5	35.5
	LocBF-FT	71.4	52.9	38.0	30.6	59.6	42.0	54.6	44.0	54.0
	AlphaEdit	7.2	4.3	6.2	4.2	24.9	12.4	44.7	21.5	2.0

Appendix

Table 5: Comparison of CrispEdit with existing methods on editing Qwen-2.5-1.5B-Instruct . Rel and gen denote reliability and generality, respectively. We perform model editing on 3,000 samples of ZsRE and evaluate editing performance with WILD framework and base performance with five representative benchmarks. Results that are the highest or within 5% of the highest results are highlighted in bold.

Method	Rel
CrispEdit (chunk size = 100)	71.1
CrispEdit (chunk size = 500, prompt masking)	12

Appendix

Table 6: Effect of prompt masking during K-FAC calculation. Even with larger number of tokens for computing K-FAC, prompt masking leads to suboptimal performance with CrispEdit - Seq .

Data	Sample Size	Edited Capabilities				Base Capabilities
		QA Context		No Context
		Rel	Gen	Rel	Gen	MMLU	IFEval	TruthfulQA	ARC-C	GSM8K
ZsRE	LLaMA-3-8B-Instruct	2.1	1.7	2.9	2.1	69.5	69.3	50.7	58.0	73.5
	No Projection (FT)	46.8	43.1	9.9	8.3	69.3	45.0	48.7	43.0	50.0
	$n=10$	53.6	48.5	10.6	9.3	69.1	48.8	50.8	42.5	57.5
	$n=50$	69.8	62.9	24.9	24.5	69.3	68.3	51.8	53.0	74.0

Appendix

Table 7: Influence of the size of capability dataset

n

on editing performances and base capability preservation. Across a range of

n

, we set

\gamma=0.9

for CrispEdit , perform model editing on 3,000 samples of ZsRE, and evaluate editing performance with WILD framework and base performance with five representative benchmarks. Results that are the highest or within 5% of the highest results are highlighted in bold. CrispEdit remains robust across a wide range of

n

. Highlighted model represents data used in Table 1 .

Data	Energy threshold	Edited Capabilities				Base Capabilities
		QA Context		No Context
		Rel	Gen	Rel	Gen	MMLU	IFEval	TruthfulQA	ARC-C	GSM8K
ZsRE	LLaMA-3-8B-Instruct	2.1	1.7	2.9	2.1	69.5	69.3	50.7	58.0	73.5
	CrispEdit ( $\gamma=0.5$ )	77.4	68.3	43.4	39.1	69.5	67.8	50.5	52.0	77.5
	CrispEdit ( $\gamma=0.6$ )	77.8	67.8	56.0	48.0	69.5	70.2	51.0	53.5	75.5
	CrispEdit ( $\gamma=0.7$ )	80.5	69.0	57.4	50.9	69.5	67.9	50.5	55.0	76.0

Appendix

Table 8: Influence of energy threshold

\gamma

on editing performances and base capability preservation. Across a range of

\gamma

, we set

n=10,000

for CrispEdit , perform model editing on 3,000 samples of three representative datasets, and evaluate editing performance with WILD framework and base performance with five representative benchmarks. Results that are the highest or within 5% of the highest results are highlighted in bold. CrispEdit remains robust across a wide range of

\gamma

Explore similar work

LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs
Oct 8, 2026Xiaobing Yu, Peijie Qiu, Jin Yang +5Language Model Editing Memory-Efficient Fine-Tuning
Yale University · Washington University in St. Louis · Icahn School of Medicine at Mount Sinai +1
CROCODIL: Cross-Model Code Editing with LLMs
Sep 3, 2026Linghan Zhong, Aditya Thimmaiah, Jayanth Srinivasa +2Reinforcement Learning LLM Post-Training
1The University of Texas at Austin · 2Cisco Research
CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing
Mar 11, 2026Manit Baser, Alperen Yildiz, Dinil Mon Divakaran +1Language Model Editing Representation Geometry in Language Models
National University of Singapore, Singapore · A*STAR Institute for Infocomm Research (A*STAR I2R), Singapore

Oct 8, 2026J/K move · Enter open · S save

LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

Xiaobing Yu, Peijie Qiu, Jin Yang, Xuanzhao Dong+4

Yale University · Washington University in St. Louis · Icahn School of Medicine at Mount Sinai+1

Language Model Editing Memory-Efficient Fine-Tuning Continual Learning LLM Compression Low-Rank Adaptation

Open paper AskPDF arXiv

Lifelong editing of LLMs requires storing thousands of edits after acquisition. A widely used family of approaches attaches one LoRA adapter per edit, which preserves behavior but grows linearly in storage. To address this challenge, we propose LadderEdit, a method that compresses each LoRA adapter after it is acquired. Each edit is first stored at low rank as a cheap sketch. We then check whether this sketch still satisfies the rewrite, generalization, and locality contract on probe prompts. Edits that pass keep the sketch; those that fail are promoted to a higher rank along a ladder until the contract is met. Because every edit retains some representation, coverage is maintained, and only hard edits consume more rank. Across ZsRE, CounterFact, and WikiBigEdit benchmarks on LLaMA-3-8B, Mistral-7B, and Qwen2.5-7B, LadderEdit tracks exact LoRA storage at 5.2x less memory and remains effective at 50,000 sequential edits.

Abstract

Explore similar work

LadderEdit: Edit-Level Residual Compression for Memory-Efficient Lifelong Editing of LLMs

CROCODIL: Cross-Model Code Editing with LLMs

CLaRE-ty Amid Chaos: Quantifying Representational Entanglement to Predict Ripple Effects in LLM Editing