Reviser: Writing and Editing Text with Autoregressive Cursor Actions

Sean Diab

cs.CLJul 23, 2026

Reviser: Writing and Editing Text with Autoregressive Cursor Actions

Authors: Sean Diab

Organizations: Independent Researcher

Abstract

Language models write text once, left to right, and models that write in other orders, such as diffusion LMs, pay for it with many full-sequence passes. We introduce Reviser, a decoder-only Transformer that writes and edits text with cursor actions (insert, move, delete, stop) that a deterministic executor applies to a canvas. Reviser is autoregressive over its own actions, so a step costs the same as an autoregressive (AR) step. We train Reviser to continue text, including a self-correction recipe that types a wrong token and deletes it, and Reviser-Edit to revise a draft, and compare each with a same-size AR model trained on the same data, using three LLM judges on continuations cut to the same length. Reviser-Edit beats its matched AR rewriter on CoEdIT SARI (+7.2 at 100M, +4.5 at 300M; two seeds agree) with a third of the decoding steps and latency, though the AR rewriter corrects grammar better. In continuation, Reviser beats diffusion LMs of similar size (62-74% win rate) at a fraction of their compute; two of three judges prefer its AR twin, and one slightly prefers Reviser at 100M. Reviser fills gaps it left and backspaces a just-typed repeated token with probability 0.85-0.87.

Figures & tables

Panel of Figure 1 — Figure 1: Inference FLOPs relative to an AR Transformer of the same size. Analytic : one shared convention, at the released decoders’ settings ( Appendix H ). Measured : counted while trained checkpoints decode at batch 1 with their released decoders, against a length-matched AR model on the same task ( Section A.2 ): c C4, released 100M Reviser; w WMT14 En–De; o OpenWebText; g GSM8K, Qwen2.5 AR. $T$ : denoising steps. FLOPs are not time: at batch 1, Dream and LLaDA ( $T{=}128$ ) take 7–8 $\times$ AR’s time, Reviser 1.2–1.7 $\times$ ( Section 9 ).

Panel of Figure 2 — Figure 2: One Reviser decoding step. The model reads only the action history and predicts one action ( Insert , Move , Delete or Stop ); a mask built from the replayed canvas removes invalid actions, including any edit of the prompt; a deterministic executor applies the action.

	Behavior			Quality			AR cut to the same length			Judges vs. AR
Model	Len	Mv/tok	Del/tok	GPT-2 $\downarrow$	Dream $\downarrow$	MAUVE $\uparrow$	GPT-2 $\downarrow$	Dream $\downarrow$	MAUVE $\uparrow$	Skywork	Codex	Claude
AR 100M	220.0	0	0	13.40	13.85	0.935	–	–	–	–	–	–
Reviser, released	67.8	0.465	0	32.33	25.68	0.811	17.80	18.85	0.921	53.9	0.200	0.158
+ run-and-jump	101.1	0.188	0	23.19	19.97	0.931	16.06	16.91	0.940	52.3	0.242	0.175
+ R&J + Delete	113.6	0.152	0.033	22.22	18.88	0.927	15.43	16.31	0.936	53.0	0.215	0.192
+ RL (3 seeds)	142.1	0.140	0.029	18.38 ∗	15.56 ∗	0.931	14.50	15.38	0.900	53.3	0.286	0.268

Table 1: C4 continuation (3,000 prompts, prompt-locked decodes, T 0.9, top-

k

50,

\leq

220 actions). Released: the random-order model of the earlier version; indented rows continue it for 608M tokens (R&J: run-and-jump). Len: mean new tokens (share ending with Stop : 99.5, 94.3, 87.9, 74.9% at 100M; 98.3, 90.6, 79.1% at 300M); Mv/Del per output token. evalPPL under GPT-2 base and Dream-7B (8 MC samples). MAUVE: standard deviation 0.004–0.008 over 10 k-means seeds. AR cut to the same length : the AR model’s text for each prompt cut to the length of that Reviser model’s text (GPT-2 tokens), scored the same way. Judges: the Reviser model’s share against the AR model of the same size, both texts cut to the same length and judged in both orders (tie

=0.5

); Codex/Claude on 300 items; full-text Skywork scores: Appendix E . 95% CIs: Skywork

\pm

1.6, Codex/Claude

\pm

0.04–0.05. RL seeds 1/2/3: Skywork 52.1/53.4/54.4, Codex 0.250/0.268/0.340, Claude 0.230/0.261/0.313. ∗ The RL reward includes a Qwen2.5-3B likelihood, so these perplexities are not independent of training. † Stored outputs of the earlier version (repeated prompt removed) vs. the stored AR 300M outputs.

Table 4

Group	Variant (change from run-and-jump)	Mv/tok	C4 PPL $\downarrow$	Gaps filled	P( Delete ) rand./rep./clean	Claude	Codex
Reference	run-and-jump, $\lambda$ = 0.82	0.205	26.40	0.401	0 / 0 / 0	0.5	0.5
Move rate	left to right only ( $\lambda$ = 1)	0	16.26	0.144	0 / 0 / 0	0.740	0.755
	$\lambda$ = 0.98	0.016	18.48	0.293	0 / 0 / 0	0.755	0.790
	$\lambda$ = 0.95	0.042	19.07	0.368	0 / 0 / 0	0.745	0.725
	$\lambda$ = 0.90	0.089	22.49	0.377	0 / 0 / 0	0.662	0.650
	$\lambda$ = 0.86	0.145	25.29	0.397	0 / 0 / 0	0.595	0.625

Table 4: Ablations at 100M (84M tokens from the released model, one run per variant). Mv/tok: cursor moves per output token on C4. C4 PPL: GPT-2 evalPPL on 1,000 C4 prompts. Gaps filled and P( Delete ) from the cursor probe (400 held-out states). Delete rows fix the wrong token immediately and mask its label unless stated. Claude/Codex: matched-length preference over the reference on 100 prompts (50 C4, 50 DCLM), both orders (

\pm

0.07). The exact rerun of the main recipe shows the run-to-run spread (

\sim

2 PPL,

\sim

0.1 judge score). Decodes without the prompt lock ( Section 3 ).

	SARI / EM $\uparrow$			ERRANT F ${}_{0.5}\uparrow$		JFLEG $\uparrow$	Reference as input kept $\uparrow$		Codex $\uparrow$
System	CoEdIT	EM	IteraTeR	Grammar	Fluency	GLEU	IteraTeR	CoEdIT	vs. AR
Copy the input	57.9	0.000	66.7	0.000	0.000	40.5	1.00	1.00	–
CoEdIT-large (T5, 770M)	68.6	0.211	58.6	0.434	0.114	59.4	–	–	–
Qwen2.5-0.5B-Instruct (0-shot)	42.4	0.001	29.2	0.253	0.025	41.2	0.02	0.06	–
Qwen2.5-1.5B-Instruct (0-shot)	38.6	0.002	22.9	0.216	0.020	48.0	0.00	0.01	–
SmolLM2-360M-Instruct (0-shot)	45.9	0.004	44.2	0.282	0.046	53.7	0.15	0.36	–

Table 5: Editing (greedy over action types; training seed 1, seed 2 in the text). SARI with one reference; EM: exact match. ERRANT F 0.5 on CoEdIT grammar items (485) and IteraTeR fluency items (88), our setup (not BEA-2019). JFLEG: GLEU on the test set with the four official references. Reference as input kept: on items whose reference differs from the input, we give the reference as the input with the same instruction and report the share returned unchanged (not measured for CoEdIT-large). Codex: preference for Reviser-Edit over the AR rewriter of the same size at matched instructions (300 items, both orders, tie

=0.5

; 95% CI

\pm

0.05). Paired SARI differences (Edit

-

AR, 95% CI): CoEdIT

+7.2\,[+6.0,+8.2]

(100M),

+4.5\,[+3.5,+5.5]

(300M); IteraTeR

-1.0\,[-3.7,+1.6]

-3.4\,[-6.0,-0.8]

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Obfuscation (target $\rightarrow$ blank)	Restoration (blank $\rightarrow$ target)
1. Initial [Hi,\|how are you?] 2. Delete [Hi\|how are you?] 3. Delete [\|how are you?] 4. $\text{{Move}}(+4)$ [how are you?\|] 5. Delete [how are you\|] 6. Delete [how are\|] 7. Delete [how\|] 8. Delete [\|]	1. Initial [\|] 2. Insert ( how ) [how\|] 3. Insert ( are ) [how are\|] 4. Insert ( you ) [how are you\|] 5. Insert ( ? ) [how are you?\|] 6. $\text{{Move}}(-4)$ [\|how are you?] 7. Insert ( Hi ) [Hi\|how are you?] 8. Insert ( , ) [Hi,\|how are you?] 9. Stop [Hi,\|how are you?]

Appendix

Table 6: Side-by-side obfuscation and restoration trajectory for a full continuation without a prefix. Restoration reverses the obfuscation and inverts each step (

\text{{Delete}}\leftrightarrow\text{{Insert}}

of the deleted token,

\text{{Move}}(\Delta)\leftrightarrow\text{{Move}}(-\Delta)

Method	Variant	$F_{\text{infer}}^{\mathcal{M}}$ ( $\mathrm{G}$ )	$I_{\text{infer}}^{\mathcal{M}}$
AR (100M)	baseline	$15.939$	1
Reviser	$p_{\text{move}}\in\{0.20,0.28,0.33\}$	$19.970,9$ – $23.900,6$	$1.25$ – $1.50$
SEDD	$T_{\text{dec}}{=}32$ (full logits each step)	$514.842,4$	$32.300,8$
SEDD	$T_{\text{dec}}{=}2048$ (full logits each step)	$32,949.915,4$	$2,067.249,7$
MDLM	$T_{\text{dec}}{=}1000$ (ancestral; full logits each step)	$16,088.825,9$	$1,009.399,3$
MDLM	$T_{\text{dec}}{=}1000$ (released cached sampler)	$1,950.013,4$	$122.342,2$

Appendix

Table 9: 100M-tier diffusion-family FLOPs multipliers under a matmul-dominant proxy. (All rows use the shared 100M reference configuration.)

Method	Variant	$F_{\text{infer}}^{\mathcal{M}}$ ( $\mathrm{G}$ )	$I_{\text{infer}}^{\mathcal{M}}$
AR (Qwen2.5-32B)	baseline	$4,099.386,9$	1
SDLM-32B-D4	$D{=}4,\;\bar{k}{=}2$ (no verify)	$8,198.941,6$	$2$
AR (Qwen2.5-7B)	baseline	$906.653,6$	1
Dream-7B	$T_{\text{dec}}{=}10$ (lin. unmask)	$8,768.930,7$	$9.671,8$
Dream-7B	$T_{\text{dec}}{=}20$ (lin. unmask)	$17,502.981,6$	$19.305$
Dream-7B	$T_{\text{dec}}{=}128$ (one token per step)	$111,830.731,1$	$123.344,5$

Appendix

Table 10: Large-scale diffusion-family FLOPs multipliers under a matmul-dominant proxy. Each block uses a size-matched AR baseline (Qwen2.5-32B, Qwen2.5-7B, or LLaMA3-8B).

Text Generation

Text generation studies methods for producing coherent, useful language from prompts, data, and dialogue context.

See all 410 papers

DEdit: Iterative Draft Editing for Speculative Decoding
Sep 29, 2026Longxuan Yu, Bingsen Chen, Peng Shi +9Efficient Language Model Inference Speculative Decoding for Diffusion Language Models
University of California, Riverside · Amazon Web Services · New York University
AutoIndexer: Flexible-Order Decoding by Training Causal Models on Chains of Edits
Oct 8, 2026Shu Ishida, Aliasghar Khani, Tianyu Zhang +3Language Model Decoding Text Generation
Autodesk Research
Self-Generated Error Training for Token Editing in Diffusion Language Models
Jun 15, 2026Lin YaoLLM Self-Correction Diffusion Model Fine-Tuning
School of Computer Science, Shanghai Jiao Tong University, Shanghai, 200240, China · Zhongguancun Academy, Beijing, 100097, China

Sep 29, 2026J/K move · Enter open · S save

DEdit: Iterative Draft Editing for Speculative Decoding

Longxuan Yu, Bingsen Chen, Peng Shi, Dongkyu Lee+8

University of California, Riverside · Amazon Web Services · New York University

Efficient Language Model Inference Speculative Decoding for Diffusion Language Models Diffusion Language Model Decoding Diffusion Language Models

Open paper AskPDF arXiv

Speculative decoding accelerates autoregressive LLMs by having a lightweight drafter propose tokens that the target model verifies in parallel. Diffusion-based drafters further reduce drafting latency by proposing multiple tokens at once. However, these tokens are predicted independently, so a single early error causes prefix verification to discard the rest of the draft, even when it contains useful downstream predictions. We introduce DEdit, a diffusion-based drafter that can not only draft by conventional parallel unmasking but also iteratively edit its draft through token-to-token predictions. Through editing, later predictions can serve as bidirectional context for repairing earlier errors and extending the accepted prefix. To teach the model to repair errors while preserving correct predictions, we propose ProposalMix, a training scheme that mixes draft predictions with ground-truth tokens based on first-pass confidence during training. Across seven benchmarks on Qwen3-4B and Qwen3-8B, DEdit achieves the highest macro-average token acceptance and speedup among the evaluated drafters, reaching macro-average speedups of

5.72\times

and

5.97\times

over autoregressive generation under greedy decoding, respectively. Further analysis shows that acceptance improves with more editing passes and wider drafting windows, and that ProposalMix halves harmful edits that shorten the accepted prefix. Moreover, restricting the editor to causal attention lowers acceptance, especially on highly predictable outputs, indicating that future context is a key source of these gains.

Abstract

Text Generation

Explore similar work

DEdit: Iterative Draft Editing for Speculative Decoding

AutoIndexer: Flexible-Order Decoding by Training Causal Models on Chains of Edits

Self-Generated Error Training for Token Editing in Diffusion Language Models