Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing

Yunlong Deng; Yan Li; Xing Hu; Yida Zhu; Sean Du; Weiyang Liu; Guangyi Chen

cs.AIOct 18, 2026

Editing as Language: Teaching LLM Agents a Grammar for Native Context Editing

Authors: Yunlong Deng, Yan Li, Xing Hu, Yida Zhu, Sean Du, Weiyang Liu, Guangyi Chen

Organizations: The Chinese University of Hong Kong · Meituan LongCat Team · Nanyang Technological University · Mohamed bin Zayed University of Artificial Intelligence ·

Abstract

A synopsis in novels folds a story into a few lines; a function in code gathers many steps with a single execution. Context editing is native to language itself. In this paper, we propose the Native Context Editing (NCE) framework, which gives language model agents a grammar for natively editing their own context. By encoding context-editing operations in a grammar of dedicated special tokens, we teach the agents to invoke edits through the language itself. We train agents on iteratively constructed trajectory segments, with each teacher edit conditioned on the evolving context. This establishes context editing as an atomic and inherent capability of LLM agents, enabling them to dynamically modify their context as they generate responses over long-horizon reasoning and interaction. Compared with context-editing methods that learn to generate tool-use code within specific external harness environments, our approach integrates context editing as an internal mechanism of the model's native generation process, enabling better generalization across diverse real-world workflows. On four long-horizon agentic benchmarks (Terminal-Bench 2.0, SWE-bench Verified, SWE-bench Multilingual, and SWE-bench Pro), NCE improves the average task success rate over the base model by 5.6 percentage points for LongCat-Flash-Lite-Sparse and by 4.5 for Qwen3.6-35B-A3B, while reducing average token usage by 36% and 15%, respectively.

Figures & tables

Figure 1: Overview of NCE. (a) The agent writes edits inline with its output, each naming a target block by its icon: drop removes a file it no longer needs, rewrite condenses a failing test log to one line; the runtime applies them before the agent continues. (b) Success vs. tokens per task for LongCat-Flash-Lite-Sparse (69B-A3B) on Terminal-Bench 2.0 (Table 2 ). NCE succeeds most with the fewest tokens at every budget.

Panel of Figure 2 — Figure 2: An abridged excerpt of an NCE training trajectory on a Go repository. (a) The rendered context at turn $t$ ; (b) the model’s output, a rationale followed by three edits and an action; (c) the blocks after execution. More detailed examples can be found in Appendix E .

Figure 3: Editing supervision from two teachers. Left: the offline teacher reads the whole trajectory and releases a block only by citing the event that ended its use (

b_{i}\subseteq b_{j}

b_{j}

supersedes or consumes

b_{i}

). Right: the online teacher reads only up to

b_{4}

and edits released blocks, here rewriting

b_{2}

and dropping

b_{3}

Figure 4: Distillation with evolving contexts. Each row supervises one turn (solid box) on the prefix left by all earlier edits (dotted box). Each editing turn closes a training sequence, so the five rows form four sequences (rows 1, 2–3, 4, and 5). Dropped blocks are drawn only for alignment.

	LongCat-Flash-Lite-Sparse (69B-A3B)						Qwen3.6-35B-A3B
	Success (%) $\uparrow$			Tokens ( $\times 10^{6}$ ) $\downarrow$			Success (%) $\uparrow$		Tokens ( $\times 10^{6}$ ) $\downarrow$
Benchmark	rules	tools	native	rules	tools	native	rules	native	rules	native
	Comp.	Tools	NCE	Comp.	Tools	NCE	Comp.	NCE	Comp.	NCE
TB 2.0	32.6	32.6	38.2 +5.6	6.62	6.38	4.60 $-$ 31%	38.2	43.8 +5.6	2.98	3.04 +2%
SWE Verified	65.2	62.4	70.0 +4.8	4.07	3.40	2.70 $-$ 34%	69.8	74.0 +4.2	2.92	2.19 $-$ 25%
SWE Multiling.	56.0	54.3	62.3 +6.3	10.83	10.93	6.12 $-$ 44%	58.0	64.3 +6.3	5.12	4.13 $-$ 19%

Table 1: Results at a 256K budget for harness compaction (Comp.), tool-call editing (Tools), and NCE ; TB 2.0 is Terminal-Bench 2.0, SWE is SWE-bench, and tokens are per task. Bold marks the best per base model. Small numbers give the success gain of NCE over the best baseline and its token change relative to harness compaction.

Budget	Comp.	Tools	NCE	Comp.	Tools	NCE
	Success (%) $\uparrow$			Tokens ( $\times 10^{6}$ ) $\downarrow$
256K	32.6	32.6	38.2 +5.6	6.62	6.38	4.60 $-$ 31%
128K	25.8	30.3	34.8 +4.5	6.08	4.96	3.70 $-$ 39%
64K	27.0	29.2	32.6 +3.4	3.42	3.58	2.52 $-$ 26%
48K	20.2	27.0	30.3 +3.3	3.40	3.17	2.15 $-$ 37%

Table 2: LongCat on Terminal-Bench 2.0 as the context budget shrinks .

Figure 5: Context management on Terminal-Bench 2.0 with LongCat under a 128K budget. (a) Success rate vs. tokens per task. (b) Context length vs. unmanaged history length (median and interquartile range); the drop of NCE at 128K is the harness compaction it keeps.

Figure 6: Building editing supervision. (a, c) Share of the 40 trajectories with at least one edit of each kind (95% bootstrap intervals). (b) Share of the original history no longer in the agent’s context at each editing decision, over 4,250 training trajectories (median and interquartile range).

Panel of Figure 7 — Figure 7: Operator use of NCE with LongCat at 256K. (a) Claude Code pools the three SWE-bench sets, and Terminus-2 runs Terminal-Bench 2.0; append removes no tokens. (b) Each operator’s removed tokens by the tool that produced the block.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Table 3 — Table 3: The dedicated tokens of NCE. Operators are shown with the chips used throughout the paper; their token strings are <op_drop> , <op_rewrite> , and <op_append> .

Panel of Figure 8 — Figure 8: The beginning of a rendered context. Each block opens with its address; the system prompt carries the lock marker. Actions, set in italics here, are assistant turns without an address. Contents are abridged with […].

Panel of Figure 9 — Figure 9: One editing turn, executed. Left: the output of NCE at turn 22 of a SWE-bench Multilingual task under Claude Code, with no reminder pending; the edits are typeset with the chips of Table 3 . Right: every block of the context before the turn and its size after it. Blocks 10, 12, 14, and 16 are earlier reads of the function that block 19 shows in its fixed form; blocks 13, 15, 17, and 18 confirm intermediate edits to it. Rewritten blocks keep their addresses; dropped addresses are retired.

Panel of Figure 10 — Figure 10: The annotation pipeline. Only the online teacher writes edits; the offline teacher decides which blocks it may touch, and the audit removes edits that fail a mechanical check.

Panel of Figure 11 — Figure 11: Teacher prompts, abridged ( […] ). Placeholders filled by the annotation driver are highlighted. The online teacher sees no turn after $t$ , no release point, and no token count.

Panel of Figure 12 — Figure 12: Annotation of one trajectory. (a) Release points of four blocks; $k_{b}$ is the turn of the cited event and $\lambda_{b}$ the last turn at which a rare literal of the block reappears. (b) The online teacher’s rationale and edits at a decision point, in the NCE grammar; block $n$ has address <sid_hi_00> <sid_lo_ $n$ > .

Statistic	Value
Segments per trajectory	$\approx$ 2.9
Decision points per trajectory	$\approx$ 2.0
milestone / soft / hard	60 / 38 / 2%
Segments without edits	31%
Edited blocks: rewrite / drop	35 / 65%
Appended notes, share of operations	2%

Appendix

Table 4: Statistics of one annotation batch. A drop that names several blocks counts once per block.

Panel of Figure 13 — Figure 13: The same two edits in NCE and in tool-call editing. NCE edits vanish once executed, while each tool call stays in the history with its result message.

Method	Solved	Success (%)	Tokens (M)	Mean	Peak	Reductions	Turns
				Context (K)
Harness compaction	23	25.8	6.08	38.3	79.7	0.6	119
Sliding window	23	25.8	4.62	43.6	71.5	1.7	92
Fixed memory state	26	29.2	5.59	44.0	72.9	1.9	95
Prompt compression	29	32.6	4.68	38.3	66.0	14.0	95
NCE	31	34.8	3.70	30.5	61.1	3.1	99

Appendix

Table 5: Five ways to manage context on Terminal-Bench 2.0 with LongCat under a 128K budget. Solved counts are out of 89 tasks; tokens, context, reductions, and turns are per task.

Trajectories (%)	One pass	Original	NCE
Uses future information	30.0	12.5	5.0
Deletes evidence used later	72.5	40.0	37.5
Edits a removed block	0.0	95.0	0.0
Restates removed text	7.5	85.0	2.5
Any defect	80.0	100.0	37.5
Defective edits (%)	4.5	22.5	1.1

Appendix

Table 6: Defects of three ways to build supervision on 40 trajectories. Original: two teachers on the original history.

Figure 14: Editing at inference. (a) Context NCE maintains when the unedited history first fills the budget on Terminal-Bench, as a share of the budget (median and interquartile range). (b) Operation mix at 256K and share of edits executed as written. (c) Cumulative share of editing turns by re-prefill time over idle time at 9.5K tokens/s; turns left of 1 hide the re-prefill entirely.

Panel of Table 7 — Table 7: Editing behavior of NCE at 256K, in percent except the two per-task counts and the tokens removed per editing turn. On Terminal-Bench, peak context reduction covers the turns before the first harness compaction.

Prefill rate	Terminus-2	Claude Code
5K tokens/s	92	62
9.5K tokens/s	98	81
20K tokens/s	100	93
40K tokens/s	100	98
Time hidden (9.5K)	98	81

Appendix

Table 8: Share of editing turns (%) whose re-prefill fits entirely in the idle time, at four prefill rates ; bold marks the rate measured from our traces. The last row gives the share of re-prefill time hidden.

Figure 15: Re-prefill time against idle time for every editing turn at 9.5K tokens/s. Points in the shaded region are hidden entirely.

Panel of Figure 16 — Figure 16: NCE keeps a Terminal-Bench task inside a 48K budget by folding finished experiments into notes. Left: context length per turn. The dashed line adds back the tokens removed by earlier edits; the shaded gap is what NCE keeps out of the context. Numbered markers are editing turns, colored by the operation that removes the most tokens. Right: rationale (italics) and edits at each editing turn, with the net tokens the turn removes. Each failed configuration becomes one line with its settings and error, and the raw logs go once a note covers them. Terminus-2 counts the unedited history and hands off at turn 40 (dotted line).

Panel of Figure 17 — Figure 17: The same model edits its context the same way under Claude Code. Both editing turns come without a reminder. The first folds a finished reproduction phase into one note with the dead ends and the root cause, then drops the superseded scripts and failing runs; the second rewrites spent file reads and edits into one-line records, keeping which fix was replaced and which one is final. Plot and excerpt conventions follow Figure 16 .

LLM Context Management

Large language model context management studies how to select, compress, retrieve, and organize information within bounded model inputs.

See all 251 papers

Agent-Editing World Model: Rethinking World Modeling for LLM Agents
Sep 23, 2026Shuang Sun, Guoxin Chen, Fanzhe Meng +6LLM World Models World Models
Gaoling School of Artificial Intelligence, Renmin University of China
ACE: Pluggable Adaptive Context Elasticizer across Agents
Jun 30, 2026Ning Liao, Zihao Long, Xiaoxing Wang +6Long-Horizon LLM Agents LLM Context Management
Meituan · Shanghai Jiao Tong University
InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing
May 28, 2025Shuaiyi Li, Zhisong Zhang, Yang Deng +6In-Context Learning Model Editing
The Chinese University of Hong Kong · City University of Hong Kong · Singapore Management University +1

Sep 23, 2026J/K move · Enter open · S save

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

Shuang Sun, Guoxin Chen, Fanzhe Meng, Jia Deng+5

Gaoling School of Artificial Intelligence, Renmin University of China

LLM World Models World Models LLM Agent Training Long-Horizon LLM Agents

Open paper AskPDF arXiv

Recent advances in large language models (LLMs) have enabled agents to tackle long-horizon tasks across diverse environments. To further improve agent performance, existing language world models typically predict environment observations, yet reconstructing high-entropy, execution-dependent tool responses offers limited value when real feedback is available. Meanwhile, agents suffer from task-state contamination, where unsupported assumptions and outdated plans persist in history and distort subsequent decisions. We propose the Agent-Editing World Model (AEWM), which models how reasoning and actions shape future task progress rather than simulating tool responses. AEWM combines Action Judge to distinguish Critical, Exploratory, and Noisy decisions with State Revision to edit noisy reasoning--action continuations from the same observed history. EditAct integrates these capabilities with real execution, directly changing the state underlying subsequent decisions rather than merely providing critiques. We train AEWM across Search, Terminal, and Software Engineering through mid-training and supervised fine-tuning. AEWM achieves 70.5% macro-F1 on our Action Judge benchmark, exceeding the strongest frontier baseline by 10.6 points. Across six benchmarks and three agent backbones, EditAct improves average scores by 3.2--6.7 points over the strongest baseline. Furthermore, rejection sampling fine-tuning on verified EditAct trajectories, termed AEWM-RFT, improves over Self-RFT by 2.2--2.6 points across three domains without online AEWM guidance.

Abstract

LLM Context Management

Explore similar work

Agent-Editing World Model: Rethinking World Modeling for LLM Agents

ACE: Pluggable Adaptive Context Elasticizer across Agents

InComeS: Integrating Compression and Selection Mechanisms into LLMs for Efficient Model Editing