DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

Tianle Chen; Pushkar Singh; Ketan Todi; Biswajit Paria; Qijia He; Wenqi Wang; Yuwen Tan; Jike Zhong

cs.AIOct 14, 2026

DeLoop: Repeated Actions as Self-Supervision for Long-Horizon Agents

Authors: Tianle Chen, Pushkar Singh, Ketan Todi, Biswajit Paria, Qijia He, Wenqi Wang, Yuwen Tan, Jike Zhong,

Organizations: Google · Boston University · University of Washington · University of Southern California ·

Abstract

Language-model agents that run long tasks often redo work they have already done: they re-issue a tool call they made earlier and get back exactly the same observation, a step that adds no new information. We identify this failure as a loop. Loops appear natively in Qwen, Llama, and Gemini agents and become far more common once the agent's history is compressed to fit its context window. To reduce these loops, we propose DeLoop, a self-supervised training method. DeLoop builds on a simple observation: the agent's own rollout can provide the supervision needed to learn from its loops. We use a looping action as a negative example and the first later non-looping action in the same rollout as a positive example. DeLoop learns from these pairs through direct preference optimization and does not require task rewards, gold answers, or LLM judges. On CodeGym under compression, DeLoop reduces the share of Qwen3-8B tasks containing loops from 57.3% to 6.1%. It also uses about one third as many steps per task, with similar task success. On Qwen3-32B, DeLoop reduces loops and steps while raising task success from 37.7% to 44.8%.

Figures & tables

Figure 1: Training an agent on its own loops breaks the loop. (Left) A repeated search and the later action that marks the server as visited form a preference pair. DPO trains the agent to prefer the recovery. (Right) Qwen3-8B on CodeGym: DeLoop and the baseline summarize at 4,096 tokens; full history is the uncompressed reference.

Figure 2: Failure rates of looping and loop-free runs. Task-failure rate for runs with a loop (red) and without one (gray), for Qwen3-8B, Qwen3-14B, and four Gemini models in their compressed settings. Detector details: Appendix E .

Figure 3: Loop rates with and without compression. (a) Qwen3-8B on CodeGym across context settings. (b) Gemini full-history runs versus compression-triggered episodes of summarization runs.

Figure 4: A self-mined preference pair. Abridged Qwen3-8B CodeGym rollout (tool names shortened; Appendix B ).

x

: history before the looping action;

y^{-}

: looping action;

y^{+}

: first later loop-breaking action.

Reference
Method	Supervision (labels needed)	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Tokens/task (k) $\downarrow$
Full history	none	29.0	31.7	12.23	24.3
Training-free context methods
Compressed baseline	none	57.3	25.1	15.00	23.9
Retrieval memory	none	42.7	27.7	13.34	26.1
ACON compressor	none	46.3	29.3	14.84	20.5

Table 1: Main comparison on CodeGym. Qwen3-8B under summarization at 4,096 tokens unless noted. Steps and estimated tokens are per-task averages. Shading: our methods; green: best compressed result; parentheses: change from the compressed baseline.

Figure 5: Model scaling on CodeGym. Qwen3-8B, Qwen3-14B, and Qwen3-32B under summarization at 4,096 tokens. Task success versus loop rate, steps, and estimated tokens per task (left to right). Squares: compressed baseline; stars: DeLoop; colors: model size.

Policy	Loop (%) $\downarrow$	Success (%) $\uparrow$
Compressed baseline	57.3	25.1
Objective (same loop signal)
DPO (DeLoop)	6.1 ( $-$ 51.2)	25.9 (+0.8)
KTO	12.6	23.8
IPO	11.7	24.0
Looping-action penalty GRPO	15.0	24.7

Table 2: Training-objective ablations. Qwen3-8B on CodeGym under summarization at 4,096 tokens. DPO, KTO, IPO, and SFT use the same 8,000 self-mined pairs; SFT uses only chosen actions. GRPO penalizes looping actions in online rollouts. Parentheses: change from the compressed baseline.

Figure 6: Cross-domain data and context management. Qwen3-8B on CodeGym. (a) Task success with and without compression. (b) Loop rate across context settings. (c) Loop rate across memory policies. Base: untrained policy.

	Loop (%) $\downarrow$		Success (%) $\uparrow$		Tokens (k) $\downarrow$
Env.	Base	Ours	Base	Ours	Base	Ours
ALFWorld	55.2	27.6 ( $-$ 27.6)	37.3	38.1 (+0.8)	9.3	6.2 ( $-$ 3.1)
Open-AgentRL	79.0	73.5 ( $-$ 5.5)	43.9	45.5 (+1.6)	–	–
AppWorld	73.7	40.9 ( $-$ 32.8)	7.0	5.8 ( $-$ 1.2)	17.9	14.1 ( $-$ 3.8)
ScienceWorld	96.5	83.5 ( $-$ 13.0)	3.5	1.5 ( $-$ 2.0)	6.5	6.0 ( $-$ 0.5)

Table 3: Evaluation in four additional agent environments. Base: untrained Qwen3-8B. Ours: DeLoop-Mix. Native metrics; green: improvement; parentheses: change from Base.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Interaction cost of looping and loop-free runs. (a) Transcript tokens and (b) steps per task for runs with a loop (red) and without one (gray), for Qwen3-8B on CodeGym and five Gemini models. Tokens use the Qwen tokenizer (Appendix A ); panel (a) uses a separate scale for Qwen3-8B.

Summaries
Context method	Loop rate (%) $\downarrow$	Success (%) $\uparrow$	Steps/task $\downarrow$	Tokens/task $\downarrow$	Fired (%)
No compression (full history)	29.0	31.7	12.23	24,319	0.0
Standard rolling summary	57.3	25.1	15.00	23,915	96.6
Instruction-ledger summary	50.0	27.0	12.40	19,535	95.7
Claude-style compact summary	50.3	26.3	13.67	20,944	95.0
LangChain-style summary	51.7	26.3	13.25	20,725	96.7

Appendix

Table 4: Context-management comparison. Qwen3-8B on 300 held-out CodeGym tasks. Fired: share of episodes in which compression fired. Steps and estimated tokens are per-task averages. The shaded row is the compressed baseline: summarization at 4,096 tokens.

Figure 8: Training objectives on the loop signal. Qwen3-8B on CodeGym under summarization at 4,096 tokens. Task success versus loop rate, steps, and estimated tokens per task (left to right). Red square: compressed baseline; blue: DeLoop; purple: KTO, IPO, and looping-action penalty GRPO.

Context method	Loop (%) $\downarrow$	Success (%) $\uparrow$
No compression (full history)	34.0	57.0
Summarization	47.1	54.7
Truncation	48.7	46.0
Claude-style compact summary	49.0	56.0
LangChain-style summary	49.7	49.7
Structured action trace	64.4	46.3

Appendix

Table 5: Compression methods on Gemini 3.5 Flash-Lite. Gemini 3.5 Flash-Lite on CodeGym. As in Figure 3 b, the loop rate counts the episodes in which compression fired (all episodes without compression); task success counts all tasks.

	Loop (%) $\downarrow$		Success (%) $\uparrow$		Steps $\downarrow$		Tokens (k) $\downarrow$
Model	Base	DeLoop	Base	DeLoop	Base	DeLoop	Base	DeLoop
Qwen3-8B	57.3	6.1	25.1	25.9	15.00	4.67	23.9	12.7
Qwen3-14B	42.9	12.3	45.1	52.3	15.28	8.18	13.3	8.6
Qwen3-32B	38.5	12.9	37.7	44.8	13.54	8.82	11.4	8.1
Qwen3.5-9B	54.7	26.0	49.7	55.3	19.15	14.95	8.0	5.8

Appendix

Table 6: Per-model CodeGym results. CodeGym under summarization at 4,096 tokens: compressed baseline (Base) and DeLoop trained on pairs mined from the same model’s rollouts. Tokens are estimated tokens per task. Shaded columns: DeLoop; green marks an improvement over the baseline.

Model	Compression	Episodes	Failure, loop (%)	Failure, no loop (%)
Qwen3-8B	Summ. 4,096	1,500	84.9	67.7
Qwen3-14B	Summ. 4,096	400	70.8	47.7
Gemini 3.5 Flash-Lite	Summ. 1,400	300	61.1	36.5
Gemini 3.1 Flash	Summ. 1,400	300	51.3	23.9
Gemini 3.6 Flash	Summ. 1,400	300	41.4	12.8
Gemini 3.1 Pro	Summ. 1,400	300	47.6	11.7

Appendix

Table 7: Failure rates behind Figure 2 . For each model’s compressed baseline on CodeGym: the compression setting, the number of episodes, and the task-failure rate of runs with and without a loop (Section 3.1 ).

Policy	A: verbatim $\downarrow$	B: assertion $\downarrow$	C: deleted $\downarrow$
Compressed baseline	15.3	17.6	43.3
DeLoop (ours)	3.4	4.5	15.0

Appendix

Table 8: Action-record probe. Re-issue rate (%) of the looping action for Qwen3-8B at 150 CodeGym looping points, with the record verbatim (A), as a one-line assertion (B), or deleted (C); 16 completions per point at temperature 0.7.

Method	Examples	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Full history	—	29.0	31.7	12.23
Compressed baseline	—	57.3	25.1	15.00
Outcome DPO	3,910	49.5	25.8	12.95
	8,000	34.7	25.7	10.99
Gold-label DPO	3,910	37.5	26.3	10.23
	8,000	22.3	25.7	8.10

Appendix

Table 10: Matched baselines at two training budgets. Qwen3-8B, CodeGym, summarization at 4,096 tokens. Each offline baseline is trained on 3,910 and on 8,000 examples; the 8,000-example rows appear in Table 1 .

Self-mined pairs	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Compressed baseline	57.3	25.1	15.00
1k	45.1	26.4	11.88
2k	34.5	24.7	9.57
3.9k	18.6	22.2	7.47
8k (DeLoop)	6.1	25.9	4.67
9.4k	6.7	23.0	4.82

Appendix

Table 11: Effect of the number of self-mined pairs. Qwen3-8B, CodeGym, summarization at 4,096 tokens; the DeLoop recipe on the number of self-mined pairs shown.

Training data	Share	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Compressed baseline	—	57.3	25.1	15.00
CodeGym only (DeLoop)	0%	6.1	25.9	4.67
+ two sources	10%	8.3	27.6	5.55
+ two sources	20%	11.9	26.8	6.74
+ two sources	37%	19.0	30.9	8.80
+ three sources (DeLoop-Mix)	37%	10.1	30.3	5.94

Appendix

Table 12: Cross-domain data mixtures. Qwen3-8B, CodeGym, summarization at 4,096 tokens; DPO on the 8,000 CodeGym pairs plus cross-domain pairs at the share shown (last row: cross-domain pairs only). The highlighted rows hold the share at 37%; the three-source row is DeLoop-Mix in Table 1 .

Method	Budget	Labels	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Tokens/task (k) $\downarrow$
Compressed baseline	—	none	57.3	25.1	15.00	23.9
DeLoop (ours)	1,000	none	45.1	26.4	11.88	22.4
Agent-R-style divergence splice	1,000	task success	52.3	24.0	13.62	27.1

Appendix

Table 13: Comparison with Agent-R-style splicing. Qwen3-8B, CodeGym, summarization at 4,096 tokens. Both trained rows use 1,000 training pairs. Labels: the supervision each method needs.

DPO $\beta$
Setting	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Compressed baseline	57.3	25.1	15.00
Default recipe (DeLoop)	6.1	25.9	4.67
$\beta=0.01$	17.3	16.3	2.06
$\beta=0.1$	17.7	23.0	6.88
$\beta=0.2$	36.7	25.0	10.57

Appendix

Table 14: Training hyperparameter ablations. Qwen3-8B, CodeGym, summarization at 4,096 tokens. Each block changes one field of the default recipe.

Mining rule (one change each)
Setting	Mined	Loop (%) $\downarrow$	Success (%) $\uparrow$	Rounds $\downarrow$
Compressed baseline	—	57.3	25.1	16.34
Default (DeLoop)	9,403	6.1	25.9	5.47
SFT (chosen action only)	—	37.7	21.3	11.71
Random later chosen	9,357	3.7	23.3	4.61
Exact-string match	9,294	7.0	22.7	5.02

Appendix

Table 15: Mining-rule and chosen-only ablations. Qwen3-8B, CodeGym, summarization at 4,096 tokens; 8,000 training pairs per row. “Mined”: pairs the rule yields. The variants pick a random later loop-breaking action, match actions as exact strings, or treat observations with similarity ratio

\geq 0.9

as the same. SFT trains on the loop-breaking (chosen) action alone. Rounds: agent turns per task.

Model	Training pairs	Loop (%) $\downarrow$	Success (%) $\uparrow$	Rounds $\downarrow$
Qwen3-8B	none	57.3	25.1	16.34
Qwen3-8B	8,000 own	6.1	25.9	5.47
Qwen3-8B	2,331 from 14B	35.0	28.7	11.28
Qwen3-14B	none	42.9	45.1	15.57
Qwen3-14B	2,331 own	12.3	52.3	8.26
Qwen3-14B	8,000 from 8B	9.3	44.3	7.21

Appendix

Table 16: Preference-pair transfer across model sizes. CodeGym, summarization at 4,096 tokens; DPO on pairs mined from the model’s own rollouts or the other model’s. Highlighted rows: DeLoop from Figure 5 . Rounds: agent turns per task.

Policy	Looping actions (% of steps) $\downarrow$	Informative repeats (% of steps)	Selectivity (%) $\uparrow$	Scored steps
Compressed baseline	24.9	10.6	29.9	13,504
DeLoop (ours)	2.5	5.8	69.8	4,204
DeLoop-Mix (ours)	2.6	7.5	74.2	5,347

Appendix

Table 17: Informative repeats and looping actions. Qwen3-8B on CodeGym under summarization at 4,096 tokens, pooled over the 300 tasks and three evaluation runs. Selectivity is the share of repeats that return new information.

Blocking and decoding
Control	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Tokens/task (k) $\downarrow$
Compressed baseline	57.3	25.1	15.00	23.9
Exact duplicate blocker	46.7	25.3	14.71	24.9
Temperature 0.7	63.3	29.3	16.99	18.5
Repetition penalty 1.2	65.7	29.0	16.85	14.7
Within-episode controls (act when a stuck detector fires)

Appendix

Table 18: Inference-time controls. Qwen3-8B on CodeGym under summarization at 4,096 tokens. Every control runs on the compressed baseline without training.

Loop signal (no labels)
Method	Reward or signal	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Tokens/ task (k) $\downarrow$	Informative groups (%)
Compressed baseline	none	57.3	25.1	15.00	23.9	—
Looping-action penalty GRPO	per-step penalty	15.0	24.7	6.39	13.4	—
+ mixed prompts	per-step penalty	18.3	28.3	7.08	13.8	—
Episode-level loop reward GRPO	one reward per episode	47.3	25.3	13.53	21.8	—
Loop-weighted SFT	weighted imitation	42.3	19.0	9.92	28.6	—

Appendix

Table 19: GRPO reward and sampling variants. Qwen3-8B on CodeGym under summarization at 4,096 tokens. Informative groups are the share of sampled groups whose rewards differ.

Main evaluation
Training sequence	SFT demonstrations	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
DPO only (DeLoop)	—	6.1	25.9	4.67
Screening evaluation: objective order
DPO only	—	6.5	16.4	4.74
DPO $\rightarrow$ SFT	solved CodeGym traces	29.7	12.5	10.79
DPO $\rightarrow$ SFT $\rightarrow$ DPO	solved CodeGym traces	2.3	21.1	4.62

Appendix

Table 20: Sequential SFT and DPO training. Qwen3-8B on CodeGym under summarization at 4,096 tokens. The lower block is a separate screening evaluation, compared only within the block.

	CodeGym			ALFWorld
Cross-domain source	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Loop (%) $\downarrow$	Success (%) $\uparrow$
Compressed baseline	57.3	25.1	15.00	55.2	37.3
SWE-agent	17.3	26.0	7.18	35.8	36.6
OpenHands	21.0	28.7	7.88	33.6	37.3
WebArena	22.7	28.3	8.00	46.3	41.8

Appendix

Table 21: Cross-domain pairs from public trajectories. Qwen3-8B on CodeGym and ALFWorld with summarization. Each shaded row trains on the self-mined CodeGym pairs plus cross-domain pairs from the named public source.

Policy	LiveCodeBench	MT-Bench	IFEval (%)
Qwen3-8B (untrained)	53/182	4.225	82.8
DeLoop (ours)	52/182	4.463	83.5
DeLoop-Mix (ours)	54/182	4.213	83.4

Appendix

Table 22: General capability evaluation. Qwen3-8B on LiveCodeBench (problems solved of 182, greedy pass@1), MT-Bench (mean judge score, 1–10), and IFEval (strict-prompt accuracy).

Memory policy	Policy	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$	Tokens/ task (k) $\downarrow$
Retrieval memory	Compressed baseline	42.7	27.7	13.34	26.1
	DeLoop-Mix (ours)	10.7	31.0	6.93	11.8
Instruction ledger	Compressed baseline	50.0	27.0	12.40	19.5
	DeLoop-Mix (ours)	7.0	25.3	5.16	8.8
Claude-style compact	Compressed baseline	50.3	26.3	13.67	20.9
	DeLoop-Mix (ours)	6.3	25.7	5.84	9.3

Appendix

Table 23: DeLoop-Mix across memory policies. Qwen3-8B on CodeGym. Each memory policy has a compressed-baseline row and a DeLoop-Mix row (shaded) on the same tasks. Tokens are estimated tokens per task (Appendix A ).

Model	Setting	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Llama-3.1-8B	Full history	44.0	17.5	17.41
	Compressed baseline	43.7	17.8	17.51
	DeLoop (ours)	2.0	19.4	5.72
Qwen3-14B	Full history	33.3	54.0	14.36
	Compressed baseline	42.9	45.1	15.28
	DeLoop (ours)	12.3	52.3	8.18

Appendix

Table 24: Other model families and sizes. CodeGym. For each model: the agent with the full history, the compressed baseline (summarization at 4,096 tokens), and DeLoop, trained on that model’s self-mined pairs and evaluated under summarization at 4,096 tokens.

	Looping points ( $n{=}150$ )			Retry points ( $n{=}80$ )
Context condition	Base	DeLoop	DeLoop-Mix	Base	DeLoop	DeLoop-Mix
Not yet executed	73.7	61.4	57.3	68.4	63.3	62.2
Proposed only	73.9	41.0	24.4	66.7	31.9	25.2
Proposed + observation	29.0	2.5	2.2	20.6	4.7	7.2
Just executed	4.9	1.9	1.9	5.6	5.6	6.9
Record present	16.5	8.3	6.8	51.7	36.1	38.0

Appendix

Table 25: Action re-issue under different context conditions. Re-issue rate (%) of the probed action under each context condition, at looping points and at retry points in Qwen3-8B CodeGym runs. Base is the untrained Qwen3-8B.

	Tasks (%)		Per action (%)		Ratio	Trained, per action (%)		Failure
Signature	Full	Comp.	Full	Comp.	( $\times$ )	DeLoop	DeLoop-Mix	OR
Exact loop (our detector)	28.6	57.0	6.6	24.8	3.77	2.3	2.5	1.62
Near-duplicate	75.7	77.9	46.7	34.3	0.73	19.0	23.7	0.68
Argument thrash	3.3	3.4	0.4	0.3	0.92	0.1	0.2	3.10
Redundant observe	52.3	56.3	21.1	22.5	1.07	6.7	7.7	0.69
Productive repeat	30.0	39.2	8.9	10.7	1.21	5.9	7.6	2.00

Appendix

Table 26: Repetition signatures under compression. Qwen3-8B on the 300 CodeGym evaluation tasks. Tasks: share of tasks with at least one event; per action: share of actions that are events, with the full history (Full) and under summarization at 4,096 tokens (Comp.); ratio: Comp./Full per action. The DeLoop and DeLoop-Mix columns are under summarization. Failure OR: odds ratio of task failure for tasks with the signature, with the full history. Rates are recomputed from the logged trajectories of three evaluation runs, and the exact-loop task shares differ slightly from the run averages in Table 1 .

Training data	Pairs	Updates	Loop (%) $\downarrow$	Success (%) $\uparrow$	Steps $\downarrow$
Compressed baseline	—	—	66.1	11.5	19.12
Random pairs	877 $\times$ 1	$\approx$ 55	54.8	11.9	13.26
Random pairs	877 $\times$ 9	$\approx$ 495	69.5	11.7	16.17
Self-mined pairs	1,003 $\times$ 8	$\approx$ 504	9.4	23.4	5.82
Self-mined pairs	4,012 $\times$ 2	$\approx$ 502	5.5	20.3	4.95
Self-mined pairs (DeLoop)	8,000 $\times$ 1	$\approx$ 500	5.5	15.6	4.80

Appendix

Table 27: Random-pair control. Qwen3-8B on another 128-task held-out CodeGym set under summarization at 4,096 tokens (Following setting from Kang et al. (2026) . Each row trains with the same DPO recipe; Pairs gives pairs

\times

epochs and Updates the number of optimizer updates. The compressed baseline is pooled over three evaluation runs.

Chosen action	Loop (%) $\downarrow$	Steps $\downarrow$
Compressed baseline (no training)	57.0	14.91
Loop-breaking action (DeLoop)	5.7	4.59
Random action from the same history	41.9	8.17
Chosen action of another pair	36.1	8.99

Appendix

Table 28: Chosen-action controls. Qwen3-8B on the 300 CodeGym evaluation tasks under summarization at 4,096 tokens. All trained rows use the same prompts, rejected actions, and DPO recipe and differ only in the chosen action. Rows are recomputed from the logged trajectories of three evaluation runs and differ slightly from the run averages in Table 1 .

Policy	Loop rate	In context	Summarized
Compressed baseline	57.3	25.2	32.1
DeLoop	6.1	3.2	2.9
DeLoop-Mix	10.1	4.3	5.8
Full history	29.0	29.0	–

Appendix

Table 29: Loops by availability of the earlier record. Loop rate (%) of Qwen3-8B on CodeGym under summarization at 4,096 tokens, averaged over three runs; the full-history agent uses no compression. A run counts under In context if it repeats a call that is still in its context, and under Summarized if every call it repeats had been summarized.

	Repeat proposals		Loop	Steps	Success
Policy	Tasks (%)	Per task	(%)		(%)
Compressed baseline	69.8	5.29	57.3	15.00	25.1
Duplicate blocker	71.3	4.77	5.7	14.71	25.3
DeLoop	13.6	0.38	6.1	4.67	25.9
DeLoop + blocker	13.7	0.25	1.3	4.84	27.7

Appendix

Table 30: Proposed repeats and executed loops with a duplicate blocker. Qwen3-8B on CodeGym under summarization at 4,096 tokens; three runs for the compressed baseline and DeLoop, one run for each blocker row. Repeat proposals : tasks in which the agent proposes a call it has already run, and such proposals per task. Loop : loops among the calls that run ( Section 3.1 ).

		Same tasks		All tasks
Model	Tasks	None	Summ.	None	Summ.
Gemini 3.5 Flash-Lite	191	40.8	47.1	34.0	36.0
Gemini 3.1 Flash	226	46.5	65.5	40.0	52.7
Gemini 3.1 Pro	139	33.1	61.9	21.3	34.3

Appendix

Table 31: Gemini loop rates on matched task sets. Loop rate (%) on CodeGym without compression (None) and with summarization at 1,400 tokens (Summ.). Same tasks : the tasks on which compression fired in the summarization run. One run per model and setting.

	Step index
Policy	0–4	5–9	10–14	15–19
Compressed baseline	4.1	22.3	32.8	42.8
DeLoop	0.8	4.1	7.0	25.7
DeLoop-Mix	1.8	3.6	4.5	7.3
Full history	2.3	6.8	11.3	13.9

Appendix

Table 32: Loop hazard by step index. Loop hazard (%): the share of steps that are looping actions among the steps an episode reaches, by step index. Qwen3-8B on CodeGym under summarization at 4,096 tokens (full history: no compression), pooled over three runs per policy.

Long-Horizon LLM Agents

Long-horizon large language model agents plan, use tools, and preserve task state across extended, multi-step interactions.

See all 451 papers

ClawLoop: Self-Improving Agents via Harness-Native Training Loop
Oct 9, 2026Hansi Yang, Linji Hao, Hongyan Hao +4LLM Agent Training Self-Improving Agents
Meituan, China · Peking University
Good to Go: The LOOP Skill Engine That Hits 99% Success and Slashes Token Usage by 99% via One-Shot Recording and Deterministic Replay
May 14, 2026Xiaohua Wang, Kai Yu, XuXiao Liang +2LLM Inference Efficiency AI Agent Reliability
1Artificial Intelligence Research Center, Bengbu Medical University, Bengbu 233000, China. · 2CHARMMIRAEL Biotech Co., Ltd, Nanjing 210000, China.
MemoryWalker: Stop Training Agents on Contexts They Never Saw
Sep 1, 2026Zinco J, Xunjie Zhu, Shen Huang +3Self-Distillation Consistency Distillation
Token Foundry, Alibaba Group

Oct 9, 2026J/K move · Enter open · S save

ClawLoop: Self-Improving Agents via Harness-Native Training Loop

Hansi Yang, Linji Hao, Hongyan Hao, An Wang+3

Meituan, China · Peking University

LLM Agent Training Self-Improving Agents LLM Agent Trajectory Synthesis LLM Agent Harnesses

Open paper AskPDF arXiv

The rapid advancement of Large Language Models (LLMs) has driven their deployment toward general-purpose harness environments, where LLM-based agents operate in sandbox workspaces and interact directly with file systems and shells. While numerous post-training pipelines exist for tool-use agents, their counterparts for harness-based agents remain scarce and are predominantly one-shot and teacher-dependent: a stronger external model synthesizes tasks and collects trajectories, the target agent is trained once, and evaluation results are consumed by human researchers rather than fed back into the loop. Such pipelines are inherently bounded by the capability of the external model and the scope of human prior, leaving no room for self-improvement. To address these limitations, we propose ClawLoop, a framework that enables self-improving without any external teacher model and with minimal prior dependencies. ClawLoop composes three phases into a self-sustaining cycle: Task Synthesis, which generates a diverse pool of training tasks via two complementary construction routines, applies diagnosis-conditioned difficulty growth followed by both agent-free and agent-based task validation; Training, which updates the agent via SFT and/or RL; and Diagnosis and Feedback, which analyzes the agent's rollout behaviors to produce a capability analysis report that will be used as feedback for task synthesis in the next iteration. Using Qwen3.6-27B as the loop model, ClawLoop yields consistent improvements on SkillsBench, Claw-Eval-Live, and ClawGym-Bench. The iteratively refined tasks are further transferred to smaller-scale Qwen3.5-4B and 9B models with gains across all benchmarks, validating data quality independently of the loop model.

Abstract

Long-Horizon LLM Agents

Explore similar work

ClawLoop: Self-Improving Agents via Harness-Native Training Loop

Good to Go: The LOOP Skill Engine That Hits 99% Success and Slashes Token Usage by 99% via One-Shot Recording and Deterministic Replay

MemoryWalker: Stop Training Agents on Contexts They Never Saw