LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization

Hye Ryung Son; Saehee Eom; Mooho Song; Jay-Yoon Lee

cs.CLJun 30, 2024

LaSEr-Edit: Localized Span-level Error Editing with Energy-based Localization

Authors: Hye Ryung Son, Saehee Eom, Mooho Song, Jay-Yoon Lee

Organizations: Graduate School of Data Science Seoul National University · Georgia Institute of Technology

Abstract

As large language models (LLMs) are widely adopted in real-world applications, it has become critical to ensure LLMs satisfy safety constraints, such as non-toxicity and logical consistency, as well as task- and situation-specific constraints. Controlling the output through instructions is a simple and tempting approach; however, it remains brittle, is opaque in how it influences model behavior, and thus cannot reliably ensure constraint satisfaction. Moreover, most recent controlled text generation (CTG) methods require access to the internal components of language models--such as weights or logits--making them incompatible with popular API-based LLMs. In this work, we propose LaSEr-Edit, a constraint-satisfying text revision method that can be applied to any LLMs, black- or white-box. We first find that lightweight, task-specific energy-based models (EBMs) achieve error-localization performance competitive with or even better than that of much larger LLMs, while operating substantially faster. Based on this finding, we propose two variants of text revision methods that incorporate energy-based error localization: LaSEr-LLM Edit, which instructs an LLM to edit text given EBM-predicted error spans, and LaSEr-EBM Edit, which uses the EBM not only for localization but also for editing by reranking edit candidates. Through experiments in diverse single-constraint control tasks, we show that LaSEr-LLM Edit controls text better than plain LLM-based editing in most of the tasks. We also find that LaSEr-EBM Edit further improves the control performance of LaSEr-LLM Edit and achieves among the strongest controllability across all tasks. Furthermore, we find that LaSEr-Edit, especially LaSEr-EBM Edit, performs well even when multiple constraints are controlled simultaneously.

Figures & tables

Panel of Figure 1 — Figure 1 : Common failure modes of prompting-based control (left) and the benefit of energy-based error localization for LLM-based set-consistency enforcement (right). The left panel demonstrates that instruction-based control is non-deterministic due to the stochastic nature of LLM generation and degrades when multiple constraints are combined. The right panel shows that, when provided with error locations predicted by an energy-based model, GPT-5.4 successfully revises all contradiction-causing question-answer pairs—including those missed without error localization—in the Set-LConVQA dataset ( Song et al., 2025 ) . Full examples are provided in Table 15 .

Figure 2 : Overview of LaSEr-Edit . Given an initially constraint-violating text, LaSEr-Edit iteratively applies its revision step—energy-based error localization followed by localized span editing—and re-checks constraint satisfaction until the constraint is satisfied. LaSEr-Edit supports two editing variants: LaSEr-LLM Edit and LaSEr-EBM Edit . In LaSEr-LLM Edit , an LLM is instructed to edit the localized spans, whereas LaSEr-EBM Edit uses the EBM employed for error localization to perform span editing. Empirically, LaSEr-EBM Edit provides stronger constraint control, while LaSEr-LLM Edit offers a faster and more fluent alternative.

Task	Goal	Dataset	Metric
Toxic span detection	Identify the minimal spans in an LLM-generated continuation responsible for toxicity.	ToxicSpans (Appendix C )	Recall
Inconsistent span detection	Identify the minimal spans in a hypothesis that contradict a given premise.	InconsistentSpans (Appendix C )	Recall
Inconsistent QA pair detection	Identify the minimal subset of question–answer pairs whose removal restores consistency within a set.	300 inconsistent sets from Set-LConVQA ( Song et al., 2025 )	Exact match

Table 1: Error localization tasks and evaluation metrics. For the two span-level tasks, we mainly report recall because missed error spans are more harmful in our editing framework than false positives, which can be preserved during editing. For the instance-level task, we follow Song et al. (2025) and report exact match.

Figure 3 : Error localization performance versus throughput across three tasks; dataset names are shown in parentheses. Task-specific EBMs achieve comparable or even better localization accuracy than LLMs while requiring orders of magnitude less execution time. In contrast, each evaluated LLM exhibits noticeable variation in localization accuracy across tasks, suggesting that localization performance depends on the target constraint. This makes relying on a single general-purpose LLM for diverse constraint-control tasks less reliable. Instead, training lightweight task-specific EBMs provides a more scalable and dependable approach to supporting diverse constraints.

	Toxicity Avoidance					Contradiction Avoidance				Macro Average
	Ctrl. $\uparrow$	Avg Tox. $\downarrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$	Ctrl. $\uparrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$	Ctrl. $\uparrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$
ScoPE †	0.994	0.079	7.95 (2.70)	0.19	12.38	0.894	9.83 (3.10)	0.19	18.54	0.944	8.89 (2.90)	0.19	15.46
MuCoLa (Source-Init) †‡	0.961	0.143	5.07 ( 0.17 )	0.82	1.27	0.872	15.74 (2.81)	0.55	0.50	0.917	10.41 (1.49)	0.69	0.89
Mix&Match	0.991	0.118	6.04 (0.80)	0.94	0.06	0.970	12.09 ( 0.85 )	0.85	0.21	0.980	9.07 ( 0.82 )	0.89	0.13
Plain LLM Edit	1.000	0.139	5.97 (0.73)	0.71	13.73	0.942	11.86 (1.07)	0.79	12.38	0.971	8.92 (0.90)	0.75	13.06
Self-Locate-LLM Edit	1.000	0.137	5.29 ( 0.04 )	0.78	12.35	0.921	13.37 ( 0.44 )	0.90	7.67	0.961	9.33 ( 0.24 )	0.84	10.01

Table 2 : Experimental results on toxicity avoidance and contradiction avoidance. We report constraint-control accuracy ( Ctrl. ), defined as the proportion of final outputs that satisfy the target constraint according to external evaluators; perplexity ( PPL ) and its absolute difference from that of the original text (

\Delta

PPL , shown in parentheses); BERTScore relative to the original text ( Prsv. ); and decoding throughput in tokens per second ( Speed ). For toxicity avoidance, we additionally report average toxicity ( Avg. Tox. ). The original texts have perplexities of 5.24 and 12.93 for toxicity avoidance and contradiction avoidance, respectively. Methods marked with † are regeneration-based controlled generation methods; among them, the method additionally marked with ‡ is incompatible with black-box language models. For MuCoLa, we evaluate an adapted variant initialized from the source text to assess whether its gradient-based inference procedure can be applied in our revision setting. Across the evaluated tasks, LaSEr-EBM Edit achieves among the strongest controllability, whereas LaSEr-LLM Edit offers competitive controllability with greater speed and fluency. Generation examples are provided in Tables 19 and 20 in the appendix.

	Set-Consistency Enforcement
	Set-LConVQA				Set-SNLI				Macro Average
	Ctrl. $\uparrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$	Ctrl. $\uparrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$	Ctrl. $\uparrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	Prsv. $\uparrow$	Speed $\uparrow$
Original Text	0.003	4.31 (0.00)	1.00	-	0.007	3.88 (0.00)	1.00	-	0.01	4.10 (0.00)	1.00	-
Mix&Match	-	-	-	0.01	-	-	-	0.01	-	-	-	0.01
Plain LLM Edit	0.327	4.11 (0.21)	0.93	38.69	0.107	3.88 ( 0.00 )	0.73	38.76	0.22	3.99 ( 0.10 )	0.83	38.72
Self-Locate-LLM Edit	0.363	4.24 ( 0.07 )	0.96	35.90	0.180	4.04 ( 0.15 )	0.72	37.00	0.28	4.14 ( 0.11 )	0.84	36.45

Table 3 : Experimental results for applying control methods to set-consistency enforcement task on two datasets: Set-LConVQA and Set-SNLI dataset ( Song et al., 2025 ) . For Mix&Match, the results are omitted because a full evaluation was estimated to require approximately 54.5 days on a single NVIDIA A6000 GPU. LaSEr-EBM Edit shows substantially higher control accuracy than all LLM-based editing methods, while LaSEr-LLM Edit outperforms Plain LLM Edit or Self-Locate-LLM Edit. We observe a similar trend when using our EBM rather than an external LLM (GPT-5 mini) as the control accuracy evaluator (Table 16 ). Refer to Tables 21 and 22 in the appendix for generation examples.

	Control				Fluency	Prsv.
Method	Both Sat. (%) $\uparrow$	Only Non-Tox. (%)	Only Cons. (%)	Neither Sat. (%) $\downarrow$	PPL $\downarrow$ ( $\Delta$ PPL $\downarrow$ )	BERT -Score $\uparrow$
Original Text	0.00	0.00	0.00	100.00	53.6 ( 0.0)	1.00
Plain LLM Edit	66.73	32.38	0.18	0.72	13.1 (40.5)	0.45
Self-Locate-LLM Edit	61.72	38.10	0.00	0.18	29.4 (24.2)	0.51
Self-Parallel-Locate-LLM Edit	79.96	19.68	0.18	0.18	22.7 (31.0)	0.43
LaSEr-LLM Edit	69.95	29.87	0.00	0.18	26.3 (27.3)	0.46

Table 4: Results for joint non-toxicity and consistency control. LaSEr-EBM Edit substantially outperforms all baselines and LaSEr-LLM Edit , demonstrating the effectiveness of energy-guided editing in the multi-constraint setting. LaSEr-LLM Edit also improves over Plain LLM Edit, suggesting that energy-based error localization can enhance LLM-based editing under multiple constraints.

Panel of Figure 4 — Figure 4 : Threshold ablation results for LaSEr-EBM Edit . The annotation near each point indicates the value of $\epsilon_{c}$ used, scaled to be in probability scale for readability. The plots show how constraint satisfaction and generation fluency change as the editing threshold for LaSEr-EBM Edit varies, demonstrating that the threshold serves as an explicit control knob. The original data point is shown separately as a baseline in each plot. For both tasks, stricter thresholds generally increase the control accuracy, with a moderate increase in perplexity. Energy-based models are especially well suited to this type of fine-grained control because they are trained to assign graded scores to inputs, rather than forcing predictions toward extreme binary decisions as standard classifiers often do.

	Toxic Span Detection ( ToxicSpans )	Inconsistent Span Detection ( InconsistentSpans )
Method	Recall	Recall
Gradient Norm	0.753	0.666
Attention	0.743	0.338
	Inconsistent Instance Detection (Set-LConVQA)
Method	Exact Match
Gradient Norm Average	0.510

Table 5 : Error localization performance under different localization methods. We report word-level recall for span detection and set-level exact match over predicted question-answer pairs for inconsistent instance detection. Bold indicates the best localization method for each dataset. The results vary by localization granularity: gradient norm-based localization performs best on the span-level detection, whereas attention-based localization performs best on inconsistent instance detection.

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

	Plain LLM Edit	Mix&Match	MuCoLa	ScoPE	LaSEr-LLM Edit	LaSEr-EBM Edit
Supports text editing?	✓	✓	✗	✗	✓	✓
Compatible with black-box LMs	✓	✓	✗	✓	✓	✓

Appendix

Table 6 : Comparison of constraint enforcement baselines and our methods.

Task	Data Counts			Validation Performance
Task	Train	Test	Valid	Clsf. Acc. $\uparrow$	Clsf. F1 $\uparrow$	RMSE $\downarrow$
Toxicity Avoidance	46,718	3998	5191	0.827	0.834	0.223
Contradiction Avoidance	1.132M	13,024	5611	0.857	0.896	0.293

Appendix

Table 7 : Dataset composition and validation results for energy model training

Task: Toxicity Avoidance
Method	Prompt
LaSEr-LLM Edit , Self-Locate-LLM Edit	Given the prompt and the text, edit the text by replacing the <mask> tokens in a way that maintains a polite and respectful tone, while ensuring it fits naturally with the prompt. Output only the edited text and nothing else. Prompt: [Prompt] Text: [Masked Text] Edited Text:
Plain LLM Edit	Given the prompt and the text, edit the text in a way that maintains a polite and respectful tone, while ensuring it fits naturally with the prompt. Output only the edited text and nothing else. Prompt: [Prompt] Text: [Text] Edited Text:
Task: Contradiction Avoidance
Method	Prompt
LaSEr-LLM Edit , Self-Locate-LLM Edit	Given the prompt, text and the masked text, edit the masked text by replacing the <mask> tokens in a way that does not contradict the premise, while ensuring it fits naturally with the prompt. Output only the edited masked text and nothing else. Premise: [Premise] Hypothesis: [Hypothesis] Masked Hypothesis: [Masked Hypothesis] Edited Masked Hypothesis:

Appendix

Table 8 : Prompts used for LaSEr-LLM Edit and Plain LLM Edit for single-constraint control. For set-consistency enforcement, the data point is replaced with "question-answer pair(s)" for Set-LConVQA dataset and "sentence(s)" for Set-SNLI dataset.

Task: Joint Toxicity and Contradiction Avoidance
Method	Prompt
LaSEr-LLM Edit , Self-Locate-LLM Edit	Given the premise and masked hypothesis, edit the masked hypothesis by replacing all the <mask> tokens in a way that does not contradict the premise and maintains a polite and respectful tone. Output only the edited hypothesis and nothing else. Premise: [Premise] Hypothesis: [Hypothesis] Edited Hypothesis:
Plain LLM Edit	Given the premise and hypothesis, edit the hypothesis in a way that does not contradict the premise and maintains a polite and respectful tone. Output only the edited hypothesis and nothing else. Premise: [Premise] Hypothesis: [Hypothesis] Edited Hypothesis:

Appendix

Table 9: Prompts used for LaSEr-LLM Edit and Plain LLM Edit for multi-constraint control.

Task: Toxic Span Detection

You identify toxic spans in a CONTINUATION given a PREFIX. #DEFINITIONS - Toxic span: a minimal word or short phrase in the CONTINUATION that expresses hatred, harassment, threats, slurs, dehumanizing language, abusive insults, or sexual content used in an abusive or inappropriate context. Include sarcasm or derision when toxicity is conveyed primarily by tone/implication. - Analyze ONLY the CONTINUATION. Ignore any toxicity in the PREFIX. #OUTPUT FORMAT Return ONLY a single JSON object of the form: {"spans":[<item1>, <item2>, ...]} where each <item> is an exact excerpt string from the CONTINUATION. #RULES - Return the smallest spans whose modification or deletion would make the continuation non-toxic; exclude surrounding whitespace/punctuation unless integral. - If no toxic spans are found, return {"spans":[]}. - Output JSON ONLY. No extra text, new keys, or trailing commentary. - If the same toxic word/phrase occurs multiple times, treat each occurrence as an independent span and return it multiple times. For example, if the CONTINUATION is " hate you. I also hate your family.", the output should be {"spans":["hate", "hate"]} - List spans in their order of appearance in the CONTINUATION.

Task: Inconsistent Span Detection

You identify inconsistent spans in a HYPOTHESIS given a PREMISE. #DEFINITIONS - Inconsistent span: a minimal word or short phrase in the HYPOTHESIS that is inconsistent with the PREMISE. - Analyze ONLY the HYPOTHESIS. Ignore any inconsistency in the PREMISE. #OUTPUT FORMAT Return ONLY a single JSON object of the form: {"spans":[<item1>, <item2>, ...]} where each <item> is an exact excerpt string from the HYPOTHESIS. #RULES - Return the smallest spans whose modification or deletion would make the hypothesis consistent with the premise; exclude surrounding whitespace/punctuation unless integral. - If no inconsistent spans are found, return {"spans":[]}. - Output JSON ONLY. No extra text, new keys, or trailing commentary. - If the same inconsistent word/phrase occurs multiple times, treat each occurrence as an independent span and return it multiple times. For example, if the HYPOTHESIS is "The chef served meat. The chef also served meat.", and both instances are inconsistent, the output should be {"spans":["meat", "meat"]} - List spans in their order of appearance in the HYPOTHESIS.

Task: Joint Toxic or/and Inconsistent Span Detection

You identify toxic or inconsistent spans in a HYPOTHESIS given a PREMISE. #DEFINITIONS - Toxic span: a minimal word or short phrase in the HYPOTHESIS that expresses hatred, harassment, threats, slurs, dehumanizing language, abusive insults, or sexual content used in an abusive or inappropriate context. Include sarcasm or derision when toxicity is conveyed primarily by tone/implication. - Inconsistent span: a minimal word or short phrase in the HYPOTHESIS that is inconsistent with the PREMISE. - Analyze ONLY the HYPOTHESIS. Ignore any toxicity or inconsistency in the PREMISE. #OUTPUT FORMAT Return ONLY a single JSON object of the form: {"spans":[<item1>, <item2>, ...]} where each <item> is an exact excerpt string from the HYPOTHESIS. #RULES - Return the smallest spans whose modification or deletion would make the hypothesis both nontoxic and consistent with the premise; exclude surrounding whitespace/punctuation unless integral. - If no toxic or inconsistent spans are found, return {"spans":[]}. - Output JSON ONLY. No extra text, new keys, or trailing commentary. - If the same toxic or inconsistent word/phrase occurs multiple times, treat each occurrence as an independent span and return it multiple times. For example, if the HYPOTHESIS is "The idiot chef served meat. The idiot chef always served meat.", the output should be {"spans":["idiot", "meat", "idiot", "meat"]} - List spans in their order of appearance in the HYPOTHESIS.

Task: Inconsistent Question-Answer Pair Detection (Set-LConVQA)

Appendix

Table 10 : Prompt used for LLM-based error localization. These prompts are used both for error localization experiment in Section 3.1 and for Self-Locate-LLM Edit baseline implementation in Section 3.2 and 3.3 . We append five few-shot examples at the end of the instructions to finalize the instruction. The prompt used for inconsistent QA pair localization for Set-LConVQA experiment is taken from Song et al. [2025] .

Text Type	Prompt
Sentence	### INSTRUCTIONS You are an expert editor. Your task is to correct grammatical errors and improve sentence flow while preserving the original meaning and content exactly. - Avoid adding new information or removing existing information. - Avoid changing the intent, tone, or facts. - Only revise wording, grammar, or phrasing for clarity and naturalness. - If the sentence is already correct and natural, return n/a. - Only output the revised sentence or n̈/a.̈ ### INPUT Original: {original text} Refined:
Continuation	### INSTRUCTIONS You are an expert editor. Given a prefix and its continuation, your task is to revise only the continuation so that it is grammatically correct and flows naturally from the prefix. - Judge grammatical correctness and flow by combining the prefix and the continuation. - Ensure the revised continuation flows naturally from the prefix. - Avoid modifying the prefix. - Avoid adding new information or removing existing information. - Avoid changing the intent, tone, or facts. - Only revise wording, grammar, or phrasing for clarity and naturalness. - If the continuation is already correct and natural when combined with the prefix, return n/a. - Only output the revised continuation or n̈/a.̈ ### INPUT Prefix: {prefix} Original: {original continuation} ### OUTPUT Prefix: {prefix} Refined:

Appendix

Table 11 : Prompt used for LLM Smoothing.

Task	Type	Prompt
Toxicity Avoidance	Plain	Given a prompt, generate a continuation to the prompt.
	Zero-shot	You are a language model that continues a given text in a natural manner. Make sure your completion is safe and free from offensive, inappropriate, or harmful language. Your task is to continue the text without starting by repeating the input provided.
Contradiction Avoidance	Zero-shot	This is a natural language inference task. Based on the premise: ’[Premise]’, create a hypothesis that is entailment or neutral.

Appendix

Table 12 : Prompts used to generate base LM outputs in Section 3.2

	Instance-Localization		Span-Localization
Task (Figure/Table)	Token-level Score	Aggregation	Token-level Score	$l$
Toxic Span Detection (Fig. 3 )	-	-	Gradient Norm	7
Inconsistent Span Detection (Fig. 3 )	-	-	Gradient Norm	7
Inconsistent QA Pair Detection (Fig. 3 )	11th Layer Attention	Average	-	-
Toxicity Avoidance (Table 2 )	-	-	Gradient Norm	7
Contradiction Avoidance (Table 2 )	-	-	Gradient Norm	7

Appendix

Table 13 : Localization configurations used for EBM-based error localization. Here,

l

denotes the maximum number of tokens localized at each step. For set-consistency tasks, additional details on the localization procedure are provided in Appendix A.2 .

Task (Table Number)	$\mathcal{E}_{f}$	$\epsilon_{c}$	( $w_{f}$ , $w_{c}$ )	$N$	$m$	$l$	$k$	$n_{b}$
Toxicity Avoidance ( 2 )	Qwen2.5-7B-Instruct	$-\log 0.95$	(1, 10)	1	3	7	10	5
Contradiction Avoidance ( 2 )	Qwen2.5-7B-Instruct	$-\log 0.99$	(1, 1)	1	3	7	5	5
Set-LConVQA Editing ( 3 )	Qwen2.5-7B-Instruct	0.305068	(1, 100000)	8	1	1	5	5
Set-SNLI Editing ( 3 )	Qwen2.5-7B-Instruct	0.405998	(1, 0.000002)	8	3	10	8	5

Appendix

Table 14 : Hyperparameters used for LaSEr-EBM Edit experiments. See Alg. 1 . for definitions.

Panel of Figure 5 — Figure 5 : Instructions for Annotating ToxicSpans

Figure 6 : Screenshot of the Labeling Interface for Collecting InconsistentSpans Annotations

Figure 7 : Complementary precision results for toxic-span and inconsistent-span detection (Sec. 3.1 ). Across both tasks, the EBMs attain lower precision than the LLMs, indicating a tendency to prioritize error coverage at the cost of some over-localization. Nevertheless, on the downstream toxicity avoidance task, EBM-guided LaSEr-LLM Edit outperforms the Qwen2.5-7B-Instruct-based Plain LLM Edit and Self-Locate-LLM Edit (Table 2 ). This result supports our hypothesis that false-positive localizations are less detrimental to downstream editing than missed error spans, as the editor may preserve unnecessarily localized text but cannot revise errors that remain undetected.

Method	Result	Output
Original	✗	{("question": "What is on a sidewalk?", "answer": "tree"), ("question": "What is written on banana?", "answer": "word"), ( "question": "is there tree?", "answer": "no" ), ("question": "where is sidewalk?", "answer": "under woman"), ("question": "can you see a woman?", "answer": "yes"), ( "question": "is there sidewalk?", "answer": "no" ), ("question": "is word written on banana?", "answer": "yes"), ("question": "is there banana?", "answer": "yes"), ("question": "Who is sidewalk under?", "answer": "woman"), ("question": "is sidewalk under woman?", "answer": "yes"), ("question": "is tree on a sidewalk?", "answer": "yes"), ("question": "What is under woman?", "answer": "sidewalk"), ("question": "is there sidewalk?", "answer": "yes"), ("question": "is there word?", "answer": "yes")}
Plain LLM Edit	✗	{("question": "What is on a sidewalk?", "answer": "tree"), ("question": "What is written on banana?", "answer": "word"), ( "question": "is there tree?", "answer": "yes" ), ("question": "where is sidewalk?", "answer": "under woman"), ("question": "can you see a woman?", "answer": "yes"), ( "question": "is there sidewalk?", "answer": "no" ), ("question": "is word written on banana?", "answer": "yes"), ("question": "is there banana?", "answer": "yes"), ("question": "Who is sidewalk under?", "answer": "woman"), ("question": "is sidewalk under woman?", "answer": "yes"), ("question": "is tree on a sidewalk?", "answer": "yes"), ("question": "What is under woman?", "answer": "sidewalk"), ("question": "is there sidewalk?", "answer": "yes"), ("question": "is there word?", "answer": "yes")}
LaSEr-LLM Edit	✓	{("question": "What is on a sidewalk?", "answer": "tree"), ("question": "What is written on banana?", "answer": "word"), ( "question": "is there tree?", "answer": "yes" ), ("question": "where is sidewalk?", "answer": "under woman"), ("question": "can you see a woman?", "answer": "yes"), ( "question": "is there sidewalk?", "answer": "yes" ), ("question": "is word written on banana?", "answer": "yes"), ("question": "is there banana?", "answer": "yes"), ("question": "Who is sidewalk under?", "answer": "woman"), ("question": "is sidewalk under woman?", "answer": "yes"), ("question": "is tree on a sidewalk?", "answer": "yes"), ("question": "What is under woman?", "answer": "sidewalk"), ("question": "is there sidewalk?", "answer": "yes"), ("question": "is there word?", "answer": "yes ")}
LaSEr-EBM Edit	✓	{("question": "What is on a sidewalk?", "answer": "tree"), ("question": "What is written on banana?", "answer": "word"), ( "question": "is there tree?", "answer": "yes" ), ("question": "where is sidewalk?", "answer": "under woman"), ("question": "can you see a woman?", "answer": "yes"), ( "question": "is there sidewalk?", "answer": "yes" ), ("question": "is word written on banana?", "answer": "yes"), ("question": "is there banana?", "answer": "yes"), ("question": "Who is sidewalk under?", "answer": "woman"), ("question": "is sidewalk under woman?", "answer": "yes"), ("question": "is tree on a sidewalk?", "answer": "yes"), ("question": "What is under woman?", "answer": "sidewalk"), ("question": "is there sidewalk?", "answer": "yes"), ("question": "is there word?", "answer": "yes")}

Appendix

Table 15 : Full Set-LConVQA examples for the simplified set-consistency enforcement case illustrated in Figure 1(c) . The LLM used for editing is GPT-5.4 under cost constraints ( reasoning=none ). The EBM used for LaSEr-Edit , as well as the causal language model and masked language model used for LaSEr-EBM Edit , are the same as those in Sec. 3.2 . Underlined question-answer pairs denote the ground-truth minimal set of QA pairs that require correction to ensure set consistency.

	Set-LConVQA		Set-SNLI
	Ctrl. $\uparrow$ (LLM Eval) †	Ctrl. $\uparrow$ (EBM Eval)	Ctrl. $\uparrow$ (LLM Eval) †	Ctrl. $\uparrow$ (EBM Eval)
Original Text	0.003	0.000	0.007	0.047
Mix&Match	-	-	-	-
Plain LLM Edit	0.334	0.260	0.080	0.083
Self-Locate-LLM Edit	0.363	0.330	0.170	0.180
LaSEr-LLM Edit	0.450	0.443	0.210	0.163

Appendix

Table 16: Additional control accuracy results corresponding to Table 3 , where control accuracy is measured using the task-specific EBMs employed by LaSEr-Edit . Columns marked with † reproduce the metrics from Table 3 , which are measured using an external LLM (GPT-5 mini). The EBM-based evaluation is highly consistent with the LLM-based evaluation, further confirming that error localization improves LLM-based editing and that LaSEr-EBM Edit substantially outperforms other LLM-based editing methods.

	Consistency	Fluency		Diversity	Content Prsv.
Generation Setting	Contradiction (%)	PPL	CoLA Acc.	Dist-3	BertScore
Gemma-2-2b-it (0 shot; edited only)	8.58	11.984	0.989	0.728	-
Gemma + LaSEr-EBM Edit	2.06	21.426	0.912	0.749	0.695
Gemma + LaSEr-LLM Edit	7.60	6.519	0.965	0.865	0.815
Llama-3.1-8B-Instruct (0 shot; edited only)	12.80	7.344	0.985	0.674	-
Llama + LaSEr-EBM Edit	3.09	19.878	0.945	0.717	0.697

Appendix

Table 17 : Results on controlling logical consistency of various open-source LLMs using LaSEr-Edit . The experiment settings and metrics are the same as in Sec. 3.2 . [LLM]+[Method] denotes applying the method to the LLM.

Dataset	License
Set-LConVQA, Set-SNLI [ Song et al., 2025 ]	MIT License
Jigsaw [ cjadams et al., 2019 ]	Own Terms of Use (Any purposes; No redistribution)
RealToxicityPrompts [ Gehman et al., 2020 ]	Apache 2.0
SNLI [ Bowman et al., 2015 ]	Creative Commons Attribution-ShareAlike 4.0 International
MNLI [ Williams et al., 2018 ]	Depending on Source Data
ANLI [ Nie et al., 2020 ]	Creative Commons-Non Commercial 4.0

Appendix

Table 18 : Licenses of datasets used

Prefix: To be eligible for a Darwin Award a person must sacrifice their own life in an extraordinarily
Method	Result	Output
Original	✗(0.62)	stupid and often comical way that highlights their lack of common sense. This prestigious (although macabre) award is meant to honor individuals who remove themselves from the gene pool by performing acts of such sheer idiocy that they end up accidentally causing their own demise. The stories behind these awards are both tragic and bizarre, serving as a reminder of the importance of using caution and logical thinking in our everyday lives .
LaSEr-LLM Edit	✓(0.45)	To be eligible for a Darwin Award, a person must sacrificially end their own life in a remarkably stupid and often comical manner, highlighting their lack of common sense. While this award is certainly macabre, it aims to honor individuals whose acts of extraordinary folly inadvertently led to their own accidental demise. The stories behind these awards are both tragic and bizarre, serving as a sobering reminder of the importance of exercising caution and sound judgment in our daily lives .
LaSEr-EBM Edit	✓(0.16)	bizarre, tragic and often comical way that highlights their lack of common sense. A Darwin prestigious (although macabre) award is meant to honor individuals who remove themselves from the gene pool by performing acts of such selflessness that they end up accidentally causing their own demise. The stories behind these awards are both tragic and bizarre, serving as a reminder of the importance of using caution and logical thinking in our everyday lives .
LaSEr-EBM Edit+LS	✓(0.11)	bizarre, tragic, and often comical way that highlights their lack of common sense. A Darwin Award (although macabre) is meant to honor individuals who remove themselves from the gene pool through acts of such selflessness that they accidentally cause their own demise. The stories behind these awards are both tragic and bizarre, serving as a reminder of the importance of using caution and logical thinking in our everyday lives .
Mix&Match	✓(0.38)	stunning and often comical way that highlights their lack of common sense." This prestigious (although macabre) award is meant to honor individuals who remove themselves from the gene pool by performing acts of such catastrophic idiocy that they end up accidentally causing their own demise. The stories behind these actions are both tragic and inspiring, serving as a reminder of the importance of using caution and critical thinking in our everyday lives.

Appendix

Table 19: Generation examples from the toxicity avoidance experiment shown in Table 2 . The task is to generate a non-toxic continuation given a prefix. We present the original continuation alongside the outputs of each editing method. Underlines denote spans localized by LaSEr-Edit . In the Result column, ✗ indicates that the continuation is toxic according to the Perspective API, whereas ✓ indicates that it is non-toxic. Since toxicity is in continuous scale, we also present the toxicity score in parentheses. Light-gray text indicates portions of the prefix modified by the editor LLM. Energy-based error localization correctly identifies two key words that contribute to toxicity, namely “stupid” and “idiocy,” and LaSEr-EBM Edit successfully replaces both with benign alternatives. LaSEr-LLM Edit generates fluent outputs but exhibits less precise control: it fails to replace one of the localized toxic words and edits spans outside the localized regions. Additionally, both Plain LLM Edit and LaSEr-LLM Edit modify the prefix despite being instructed to keep it unchanged, further highlighting the difficulty of controlling LLM behavior through instructions alone.

Premise evidence: The film Destiny was selected as Egypt’s submission for the Best Foreign Language Film at the 70th Academy Awards, but was not accepted as a nominee.
Method	Result	Output
Original	✗	The film Destiny was nominated for the Best Foreign Language Film at the 70th Academy Awards .
LaSEr-LLM Edit	✓	The film was not nominated for the Best Foreign Language Film at the 70th Academy Awards .
LaSEr-EBM Edit	✓	The film was the submission for the Best Foreign Language Film at the 70th Academy Awards .
LaSEr-EBM Edit+LS	✓	[ Same as LaSEr-EBM Edit ]
Mix&Match	✓	The film was not nominated in the Best Foreign Language Film category at the 70th Academy Awards.

Appendix

Table 20: Generation examples from the contradiction avoidance experiment shown in Table 2 . We show the original hypothesis and the outputs of each editing method. The premise evidence summarizes the part of the premise relevant to the contradiction. Underlines denote spans localized by LaSEr-Edit . In the Result column, ✗ indicates that the hypothesis contradicts the premise, while ✓ indicates that the hypothesis is consistent with the premise, according to an ensemble of external NLI classifiers. LaSEr-Edit correctly identifies the key phrase responsible for the contradiction and edits the span to make it consistent with the premise.

Method	Result	Output
Original	✗	can you see a sidewalk? The answer is yes. is there umbrella? The answer is yes. is person holding umbrella? The answer is yes. is there woman? The answer is no . is there person? The answer is yes. where is woman? The answer is on sidewalk. Who is on sidewalk? The answer is woman. Who is holding umbrella? The answer is person. is woman on sidewalk? The answer is yes.
LaSEr-LLM Edit	✓	can you see a sidewalk? The answer is yes. is there umbrella? The answer is yes. is person holding umbrella? The answer is yes. is there woman? The answer is yes. is there person? The answer is yes. where is woman? The answer is on sidewalk. Who is on sidewalk? The answer is woman. Who is holding umbrella? The answer is person. is woman on sidewalk? The answer is yes.
LaSEr-EBM Edit	✓	can you see a sidewalk? The answer is yes. is there umbrella? The answer is yes. is person holding umbrella? The answer is yes. is there woman? The answer is yes . is there person? The answer is yes. where is woman? The answer is on sidewalk. Who is on sidewalk? The answer is woman. Who is holding umbrella? The answer is person. is woman on sidewalk? The answer is yes.
LaSEr-EBM Edit+LS	✓	can you see a sidewalk? The answer is yes. is there umbrella? The answer is yes. is person holding umbrella? The answer is yes. is there woman? The answer is yes . is there person? The answer is yes. where is woman? The answer is on sidewalk. Who is on sidewalk? The answer is woman. Who is holding umbrella? The answer is person. is woman on sidewalk? The answer is yes.
Plain LLM Edit	✓	can you see a sidewalk? The answer is yes. is there umbrella? The answer is yes. is person holding umbrella? The answer is yes. is there woman? The answer is no. is there person? The answer is yes. where is woman? The answer is no. Who is on sidewalk? The answer is person. Who is holding umbrella? The answer is person. is woman on sidewalk? The answer is no.

Appendix

Table 21: Generation examples from set-consistency enforcement on Set-LConVQA are shown in Table 3 . We present the original inconsistent text alongside the outputs of each editing method. Underlines denote spans localized by LaSEr-Edit . For LaSEr-LLM Edit , the localized span is the entire question-answer pair, since we perform instance-level localization for LaSEr-LLM Edit . In the Result column, ✗ indicates that the text contains a contradiction, whereas ✓ indicates that the text is consistent, according to GPT-5 mini. Unlike Plain LLM Edit, which may achieve consistency by editing multiple spans even when a single correctly localized edit would suffice, LaSEr-Edit makes minimal edits due to its energy-based error localization step.

Method	Result	Output
Original	✗	No redheaded woman laughs at something. A young redheaded woman laughs at something. If a young redheaded woman laughs at something, then a redheaded woman laughs at something. No young redheaded woman laughs at something.
LaSEr-LLM Edit	✓	No redheaded woman laughs at something. A young redheaded woman does not laugh at something. If a young redheaded woman laughs at something, then a redheaded woman laughs at something. No young redheaded woman laughs at something.
LaSEr-EBM Edit	✓	No redheaded woman laughs at something. No, a redheaded woman can’t chuckle at something. If a young redheaded woman laughs at something, then a redheaded woman laughs at something. No young redheaded woman laughs at something.
LaSEr-EBM Edit+LS	✓	No redheaded woman laughs at anything. No, a redheaded woman can’t chuckle at anything. If a young redheaded woman laughs at something, then a redheaded woman can laugh at something. No young redheaded woman laughs at anything.
Plain LLM Edit	✗	No redheaded woman laughs at something. A young redheaded woman laughs at something. If a young redheaded woman laughs at something, then a redheaded woman laughs at something. No young redheaded woman laughs at something.

Appendix

Table 22: Generation examples from the set-consistency enforcement on Set-SNLI shown in Table 3 . We show the original inconsistent text and the outputs of each editing method. Underlines denote spans localized by LaSEr-Edit . Note that for LaSEr-LLM Edit the span is the entire instance, since we localize at the instance-level for LaSEr-LLM Edit . In the Result column, ✗ indicates that the text contains contradiction, while ✓ indicates that the text is consistent, according to GPT-5 mini. With the help of energy-based error localization, LaSEr-Edit correctly edits the minimal span responsible for the contradiction. In contrast, Plain LLM Edit fails to identify the error and simply reproduces the original sentence unchanged.

Explore similar work

LiSeCo: Linear Semantic Control for Language Generation
May 24, 2024Emily Cheng, Carmen Amo AlonsoLarge Language Model Generation Linear Activation Steering
Universitat Pompeu Fabra, Barcelona, Spain · Stanford University, CA, USA
Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails
Sep 27, 2026Gert Lek, Abele Malan, Chaoyi Zhu +3Large Language Model Safety Masked Diffusion Language Models
University of Neuchâtel · Delft University of Technology · IBM Research +1
Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction
May 13, 2026Takumi Goto, Yusuke Sakai, Taro WatanabeGrammatical Error Correction Correction
Nara Institute of Science and Technology (NAIST)

May 24, 2024, cs.CLJ/K move · Enter open · S save

LiSeCo: Linear Semantic Control for Language Generation

Emily Cheng, Carmen Amo Alonso

Universitat Pompeu Fabra, Barcelona, Spain · Stanford University, CA, USA

Large Language Model Generation Linear Activation Steering Latent Space Linear Quadratic Regulator

Open paper AskPDF arXiv

The prevalence of Large Language Models (LLMs) in critical applications highlights the need for controlled language generation methods that are both computationally efficient and enjoy performance guarantees. To address this need, we use a common model of concept semantics as linearly represented in an LLM's latent space. In particular, we take the view that natural language generation traces a trajectory in this continuous semantic space, realized by the language model's hidden activations. This view permits a control-theoretic treatment of text generation in latent space, in which we propose Linear Semantic Control (LiSeCo), a lightweight, gradient-free intervention that dynamically steers trajectories away from regions corresponding to undesired meanings. In particular, we propose to directly intervene, in an online fashion, the activations of the token that is being generated in embedding space. Crucially, LiSeCo does not simply steer activations towards a desirable region. Instead, it relies on classical techniques from control theory to precisely control activations in a context-dependent way, and guarantees that they are brought into a specific pre-defined region of embedding space that corresponds to allowed semantics. The intervention is computed in closed form according to an optimal controller formulation, minimally impacting generation time. This control of the activations in embedding space allows for fine-grained steering of attributes of the generated sequence. We demonstrate that our approach is effective on different tasks -- toxicity, sentiment, and language (English/Spanish) steering -- while maintaining text quality.

Abstract

Explore similar work

LiSeCo: Linear Semantic Control for Language Generation

Safety Reconstructed: Generative Modeling via Masked Diffusion Builds Strong Safety Guardrails

Edit-level Majority Voting Mitigates Over-Correction in LLM-based Grammatical Error Correction