Software development is iterative, yet agentic coding benchmarks hide design issues through their single-shot setup. Recent iterative benchmarks attempt to remedy this but heavily constrain an agent's design decision space, making it impossible to faithfully measure how their decisions shape future extensions. We introduce SlopCodeBench, a benchmark of 36 problems and 196 checkpoints where agents repeatedly extend their own solutions. Unlike prior iterative benchmarks, our evolving specifications demand architectural decisions but leave internal structure to the agent. We measure two forms of degradation: structural erosion (concentrated complexity) and verbosity (redundant code). Evaluating 15 coding agents across open and closed models, we find that no agent fully solves any problem end-to-end, and the best agent passes 14.8% of checkpoints. Quality degrades across checkpoints, with structural erosion rising in 77% of trajectories and verbosity in 75.5%. Compared to 473 open-source Python repositories, agent code is 2.3x more verbose and 2.0x more eroded, and the human repositories degrade less often and by smaller margins across their git histories. Explicit quality guidance reduces initial verbosity and erosion by up to a third, without affecting degradation rates. SlopCodeBench provides the first measurement of code degradation under iterative extension, revealing that agents pass checkpoints while producing code that erodes and bloats with each turn.
Figures & tables
Figure 1: Iterative evaluation in SCBench. A task unfolds over checkpoints. At each step, the agent extends its prior workspace to satisfy an updated spec (orange). Correctness, erosion, and verbosity are scored externally and withheld from the agent (gray). Local shortcuts compound: correctness falls from 1.00 to 0.10 while erosion and verbosity grow.
Solve Rate (%)
Cost & Time
Quality
Model
Strict
Iso.
Core
Partial
$/CKPT
Net $
Min/CKPT
Erosion
Verbosity
Opus 4.5
9.2
17.3
57.7
25.0
2.53 ± 1.57
492.40
7.2 ± 3.7
0.70 ± 0.20
0.43 ± 0.18
Opus 4.6
9.7
20.9
67.3
27.8
3.17 ± 2.82
620.80
13.1 ± 12.9
0.75 ± 0.15
0.44 ± 0.19
Opus 4.7
8.2
20.9
65.8
36.1
2.17 ± 1.83
425.34
6.8 ± 5.0
0.76 ± 0.17
0.48 ± 0.21
Sonnet 4.6
7.1
16.8
57.7
27.8
1.96 ± 1.97
383.83
14.2 ± 14.2
0.75 ± 0.20
0.44 ± 0.19
GPT 5.2 Codex
9.7
21.9
56.1
36.1
0.85 ± 0.80
167.29
12.3 ± 9.7
0.73 ± 0.18
0.50 ± 0.16
Table 1: Per-model performance on SlopCodeBench. For each model we report its best just-solve run, ranked by isolated solve rate with core, strict, and $/checkpoint as tiebreakers. Solve rates use a fixed 196-checkpoint denominator; missing checkpoints count as unsolved. Other metrics are only for ran checkpoints. Bold marks the best value per column. See Table 8 for the full set of runs.
Figure 2: Performance degrades as problems progress and cost per checkpoint steadily rise. Left: Checkpoint solve rate by type. Right: Mean cost per checkpoint as problems progress.
Figure 3: Models from both frontier labs produce similar degradation trends as the problem progresses . Erosion and verbosity across problem progress for six representative models.
Figure 4: Agents produce lower quality code and degrade it faster than real developers. Verbosity and erosion across normalized trajectory progress, with 95% confidence intervals. GitHub-star bins over 473 repositories for a total of 13667 checkpoints. Table 2 has HEAD commit details.
Group
n
Verbosity
Erosion
Hobby ( <100★ )
114
0.21±0.14
0.33±0.25
Niche ( 100–1k★ )
124
0.19±0.11
0.32±0.22
Established ( 1k–10k★ )
120
0.18±0.08
0.33±0.20
Major ( ≥10k★ )
115
0.19±0.10
0.37±0.19
Human panel (all)
473
0.19±0.11
0.34±0.22
Agent checkpoints
2869
0.44±0.18
0.68±0.20
Table 2: Verbosity and erosion at HEAD commit, by GitHub-star tier and against agent checkpoints.
Figure 5: Prompting can improve initial quality, but does not remove the iterative issues. Mean structural erosion (top) and verbosity (bottom) at each progress bin, per model.
Model
Prompt
Strict
Iso.
Core
Erosion
Verbosity
Cost
GPT 5.3 Codex
Just-Solve
11.2
26.0
60.7
0.64
0.46
0.66
Anti-Slop
11.2 (+0.0)
23.5 ( -2.6 )
57.1 ( -3.6 )
0.42 ( -0.22 )
0.31 ( -0.15 )
0.73 ( +0.07 )
Plan-First
8.2 ( -3.1 )
21.4 ( -4.6 )
62.2 ( +1.5 )
0.55 ( -0.10 )
0.40 ( -0.06 )
0.68 ( +0.02 )
GPT 5.4
Just-Solve
10.7
23.5
62.8
0.51
0.33
0.72
Anti-Slop
9.2 ( -1.5 )
25.5 ( +2.0 )
61.2 ( -1.5 )
0.28 ( -0.23 )
0.24 ( -0.09 )
0.90 ( +0.18 )
Plan-First
9.7 ( -1.0 )
25.0 ( +1.5 )
63.3 ( +0.5 )
0.39 ( -0.11 )
0.30 ( -0.02 )
0.83 ( +0.11 )
Table 3: Raw values per prompt with the lift over just-solve in parentheses. Green means improvements while red indicates regressions.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Problem
CPs
Design decisions stressed
cfgpipe
6
Source-priority resolution, typed parsing boundaries, nested schema representation, watch and event architecture, redaction, and store-prefix composition.
circuit_eval
8
Circuit IR, parser separation across formats, two-valued versus three-valued logic, optimization pass pipeline, and equivalence checking.
code_search
5
Regex versus AST matching, metavariable representation, rewrite conflict handling, selector semantics, and language adapter boundaries.
dag_execution
3
DAG DSL parsing, task parameter model, execution ordering, cache key design, and per-task cache overrides.
database_migration
5
Migration operation model, rollback representation, dependency sorting, cycle errors, and constraint or index handling.
datagate
7
Dataset ingestion abstraction, query and filter pipeline, pagination and export contracts, cache invalidation, runtime config, and access control.
Appendix
Table 4: SlopCodeBench problems and the design decisions stressed by iterative checkpoints, part 1 of 2. CPs gives the number of checkpoints.
Problem
CPs
Design decisions stressed
forge
8
Resource models for blueprints, allocations, units, modules, revision gates, strict JSON contracts, and admin access boundaries.
Layer merge semantics, conflict resolution, fragment expansion, environment interpolation, multi-run manifests, and JSON Schema validation.
log_query
5
SQL-like grammar design, NDJSON streaming execution, aggregation and joins, schema label mapping, and subquery scope.
meshctl
8
Declarative resource specs, persisted state shape, lifecycle and topology operations, security model, migrations, network exposure, and multi-region policy.
metric_transform_lang
5
DSL parser and interpreter design, stream processing, aggregation and window state, temporal joins, and resumable execution.
Appendix
Table 5: SlopCodeBench problems and the design decisions stressed by iterative checkpoints, part 2 of 2.
Metric
Pass
Cost
Reported erosion
−0.018
0.167
Best variant (CC > 12 + Linear SLOC)
−0.136
0.095
LOC
−0.212
0.502
Mean CC
−0.046
0.218
Max CC
−0.117
0.356
Clone Ratio
−0.042
0.114
Appendix
Table 6: Erosion metric sensitivity sweep.
Model
Harness
Version
Reasoning
Opus 4.5
Claude Code
2.0.51
high
Opus 4.6
Claude Code
2.1.32
high
Sonnet 4.6
Claude Code
2.1.44
high
GPT 5.2 Codex
Codex CLI
0.93.0
high
GPT 5.3 Codex
Codex CLI
0.98.0
high
GPT 5.3 Spark
Codex CLI
0.100.0
high
Appendix
Table 7: Harness version and reasoning effort used for each model in the main evaluation.
Solve Rate (%)
Cost
Quality
Model
Harness
Version
Think
Prompt
Strict
Iso.
Core
Partial
$/CKPT
Net $
Erosion
Verbosity
Opus 4.5 [ 5 ]
Claude Code
2.0.51
High
Just Solve
9.2
17.3
57.7
25.0
2.53 ± 1.57
492.40
0.70 ± 0.20
0.43 ± 0.18
Opus 4.6 [ 6 ]
Claude Code
2.1.32
High
Just Solve
9.7
20.9
67.3
27.8
3.17 ± 2.82
620.80
0.75 ± 0.15
0.44 ± 0.19
Opus 4.7 [ 7 ]
Claude Code
2.1.111
High
Just Solve
8.2
20.9
65.8
36.1
2.17 ± 1.83
425.34
0.76 ± 0.17
0.48 ± 0.21
Sonnet 4.6 [ 8 ]
Claude Code
2.1.44
High
Just Solve
7.1
16.8
57.7
27.8
1.96 ± 1.97
383.83
0.75 ± 0.20
0.44 ± 0.19
GPT 5.2 Codex [ 45 ]
Codex CLI
0.93.0
High
Just Solve
9.7
21.9
56.1
36.1
0.85 ± 0.80
167.29
0.73 ± 0.18
0.50 ± 0.16
Appendix
Table 8: Full set of runs on SlopCodeBench across all harnesses, versions, reasoning levels, and prompt strategies. Bold marks the best value per column.