Super Library Agent: Joint Generation and Maintenance of Multiple Applications Beyond the Single Codebase

Daegyu Sung; Yukyeong Lee; Geon Park; Yumin Choi; Sung Ju Hwang

cs.SEAug 29, 2026

Super Library Agent: Joint Generation and Maintenance of Multiple Applications Beyond the Single Codebase

Authors: Daegyu Sung, Yukyeong Lee, Geon Park, Yumin Choi, Sung Ju Hwang

Organizations: KAIST

Abstract

Organizations often develop and maintain portfolios of related applications: independently deployable codebases that share substantial domain logic, interface patterns, or operational conventions. As LLM coding agents are increasingly used to generate and maintain such software, a naive application-by-application workflow duplicates shared logic across codebases and allows prolonged agentic maintenance to accumulate verbosity, dead code, and structural erosion. We introduce the Super Library Agent problem, where an agent sequentially generates a portfolio of N related applications while maintaining a shared Super Library of reusable cross-application components. A minimal sequential scaffold can in principle extract shared code and migrate applications to the evolving library, but in practice suffers from low extraction recall and fragile dependency migration. We address these failures with candidate-guided extraction over code chunk summaries, pre-extraction codebase consolidation, and context-aware migration using extraction traces and call-graph information. Across WebGen-Bench and PaperBench, our method preserves application functionality while significantly reducing redundancy and token footprint (verbosity, token length) over zero-shot, and avoiding the structural erosion introduced by naive library construction, with additional reductions in LOC and MDL. Our code is available at https://github.com/sbigstar0310/super-library-agent.

Figures & tables

Figure 1: Super Library Agent (SLA) problem. Independent generation duplicates shared logic; SLA maintains a shared library across a sequence of codebases.

Figure 2: Overview of the proposed workflow. Minimal scaffold SLA-Naive (left) vs. SLA-Full (right).

	Functionality		Maintainability					Library Size
Method	Acc. $\uparrow$	Appr. $\uparrow$	LOC $\downarrow$	Tok $\downarrow$	MDL $\downarrow$	Eros. $\downarrow$	Verb. $\downarrow$	LOC	MDL
WebGen-Bench
Zero-Shot	76.04	3.84	9393	70364	34875	0.1032	0.1603	–	–
Librarian ( $K{=}1$ )	75.09	3.90	9202	69492	34566	0.1266	0.1558	184	576
Librarian ( $K{=}8$ )	75.78	3.89	8973	68159	33919	0.1352	0.1472	222	639
Naive-Implicit	76.95	3.89	9133	69777	35103	0.1567	0.1408	447	1212

Table 1: Initial portfolio construction results. Results are averaged over three 8-task WebGen-Bench suites and five 4-task PaperBench suites, each with three trials. PaperBench Acc. denotes the rubric-based Code-Dev Score, and appearance is not applicable. Maintainability metrics cover the complete portfolio, while Library Size covers only the Super Library. Librarian

K{=}1

omits best-of-

K

selection.

	Patch Size			Functionality
Method	Total $\downarrow$	App $\downarrow$	Lib	Orig. $\uparrow$	Req. $\uparrow$	Appr. $\uparrow$
Zero-Shot	936	936	0	77.9	81.9	3.93
Librarian	522	500	22	74.0	76.9	3.83
Naive-Implicit	632	618	14	76.9	82.9	3.82
Naive-Ward	380	363	18	77.7	81.1	3.95
SLA-Full	256	232	24	77.4	80.0	3.86

Table 2: Maintenance under shared policy updates. Results are averaged over three WebGen-Bench suites and three trials. Patch Size counts source lines added by the patch. Orig. and Req. report post-patch pass rates on the original and requested behaviors. Appr. denotes appearance quality.

Table 3: Ablations on WebGen-Bench. We report the mean values over three suites

\times

three trials. Maintainability metrics are computed over the final portfolio including the shared library.

Figure 3: LOC dynamics across sequential rounds. Columns are WebGen suites 1–3. In (a) and (b), each colored line is one application. Values are relative to each app’s first round; shaded area shows

\pm 1

std over three trials.

Panel of Figure 4 — Figure 4: Categories of highly reused Super Library symbols (imported by $\geq 4$ of 8 apps). SLA-Full captures more behavioral and page/domain abstractions beyond primitives.

	No. of Apps			Total Exports
Method	1–2	3–5	6–8	Total Exports
Librarian	1.2	1.6	1.0	4.2
Naive-Implicit	1.3	3.6	2.7	8.2
Naive-Ward	1.4	3.3	2.0	7.2
SLA-Full	2.2	6.0	4.8	13.3

Table 4: Super Library utilization, averaged over three suites

\times

three trials.

Metric	w/o library	w/ library		$\Delta$
		App	Lib
UI acc. (%) $\uparrow$	80.95	84.35		$+3.40$
Appear. (0–5) $\uparrow$	3.83	3.92		$+0.09$
LOC $\downarrow$	8,256	6,567	750	$-940$
MDL $\downarrow$	29,291	27,442	1,985	$+135$

Table 5: Library-as-prior ablation. 8 tasks

\times

3 trials.

\Delta

on total (App + Lib).

Appendix figures & tables20 assets

Supplementary material from the paper’s appendix.

Appendix

	vs. Zero-Shot		vs. Librarian		vs. Naive-Implicit		vs. Naive-Ward
Metric	$\Delta$ [95% CI]	$p$	$\Delta$ [95% CI]	$p$	$\Delta$ [95% CI]	$p$	$\Delta$ [95% CI]	$p$
WebGen-Bench ( $n=24$ )
Acc. $\uparrow$	-0.85 [-3.24, +1.38]	0.484	-1.14 [-3.07, +0.78]	0.274	+0.23 [-2.38, +2.60]	0.864	-1.04 [-3.23, +1.06]	0.358
Appear. $\uparrow$	-0.028 [-0.125, +0.042]	0.539	+0.028 [-0.069, +0.125]	0.575	+0.028 [-0.056, +0.125]	0.575	+0.042 [-0.028, +0.125]	0.328
LOC $\downarrow$	+105 [+54, +154]	0.001	+53 [+0, +102]	0.059	+73 [+28, +126]	0.009	+29 [+3, +56]	0.044
Tok $\downarrow$	+591 [+275, +891]	0.001	+316 [-1, +633]	0.068	+518 [+232, +884]	0.006	+268 [+60, +492]	0.028

Appendix

Table 6: Paired significance tests comparing SLA-Full against each baseline (protocol in the text).

\Delta

is the compared method minus SLA-Full , so SLA-Full is favored by a positive

\Delta

on lower-is-better (

\downarrow

) metrics and by a negative

\Delta

on higher-is-better (

\uparrow

) ones; bold marks

p<0.05

. WebGen LOC/Tok/MDL are per-task.

Suite	WebGen-Bench tasks
S1 Content Presentation Brochures & Portfolios	1. [000019] SW solutions brochure. 2. [000075] Team projects & skills. 3. [000015] Clinical office info. 4. [000016] Solar company products. 5. [000040] Consulting & training. 6. [000072] Personal CV/resume. 7. [000071] Designer portfolio. 8. [000064] Shipping news blog.
S2 Data Management CRUD / ERP / CRM	1. [000085] Construction project mgmt. 2. [000022] Customer-service call log. 3. [000046] Hospital admin & inventory. 4. [000083] Detective agency case mgmt. 5. [000047] E-government office system. 6. [000044] Supply-chain traceability. 7. [000023] CRM sales lead tracking. 8. [000080] Advanced to-do list app.
S3 User Interaction Social & Job Platforms	1. [000052] Blue-collar job board. 2. [000090] Social networking site. 3. [000051] Driver recruitment listings. 4. [000077] Engineer tool info site. 5. [000092] Skill-sharing platform. 6. [000053] Student internship portal. 7. [000091] Matrimonial matching site. 8. [000027] Promotions & discounts forum.

Appendix

Table 7: WebGen-Bench evaluation suites. We cluster the 101 WebGen-Bench tasks with Ward agglomerative clustering (

k{=}14

, Euclidean) over text-embedding-3-small embeddings of each task’s instruction, category metadata, and UI-test interactions, and select three of the resulting clusters as evaluation suites. From each selected cluster we take the eight tasks closest to the centroid; each suite is then used as a sequential application stream.

Suite	PaperBench tasks
S1	1. fre : Functional Reward Encodings. 2. rice : RICE. 3. sapg : Split and Aggregate Policy Gradients. 4. ftrl : RL Fine-tuning as Forgetting Mitigation.
S2	1. robust-clip : Robust CLIP. 2. sample-specific-masks : Sample-specific Masks. 3. lca-on-the-line : OOD Generalization. 4. test-time-model-adaptation : Test-Time Adaptation.
S3	1. mechanistic-understanding : DPO and Toxicity. 2. what-will-my-model-forget : Forgotten Examples. 3. stay-on-topic-with-classifier-free-guidance : CFG. 4. adaptive-pruning : Adaptive Pruning and Tuning.
S4	1. bridging-data-gaps : Diffusion Transfer Learning. 2. sequential-neural-score-estimation : Neural Score Estimation. 3. stochastic-interpolants : Data-Dependent Couplings. 4. all-in-one : Simulation-Based Inference.
S5	1. bam : Batch-and-Match Variational Inference. 2. pinn : Training PINNs Loss Landscapes. 3. lbcs : Refined Coreset Selection. 4. bbox : BBox-Adapter for Black-Box LLMs.

Appendix

Table 8: PaperBench evaluation suites. We construct five disjoint suites of four tasks each. Suites S1–S3 are topically coherent (RL, VLM, NLP), S4 is a diffusion/score-based inference suite, and S5 is a heterogeneous low-reuse suite included as a robustness probe. Each suite is used as a sequential task stream for evaluating Super Library Agent methods.

Metric	Suite	Zero	Librarian	Naive		Full
				Implicit	Ward
Functionality
Acc. $\uparrow$	S1	82.99 $\pm$ 2.12	81.97 $\pm$ 1.56	83.67 $\pm$ 0.00	83.67 $\pm$ 3.68	83.33 $\pm$ 2.12
	S2	70.28 $\pm$ 2.55	70.83 $\pm$ 2.21	70.83 $\pm$ 3.00	70.00 $\pm$ 4.64	72.22 $\pm$ 3.37
	S3	74.85 $\pm$ 0.52	74.55 $\pm$ 1.82	76.36 $\pm$ 1.57	74.54 $\pm$ 3.96	76.06 $\pm$ 2.62
	Avg.	76.04 $\pm$ 5.82	75.78 $\pm$ 5.17	76.96 $\pm$ 5.83	76.07 $\pm$ 7.00	77.21 $\pm$ 5.44

Appendix

Table 9: Per-suite results on WebGen-Bench. We report functionality and maintainability metrics across three disjoint suites of eight tasks each, as mean

\pm

std over three independent trials per suite. Avg. is pooled across all 9 trials (3 suites

\times

3 trials).

Metric	Suite	Zero	Librarian	Naive		Full
				Implicit	Ward
Functionality
Score $\uparrow$	S1	0.3409 $\pm$ 0.0918	0.3539 $\pm$ 0.0476	0.3393 $\pm$ 0.0822	0.3918 $\pm$ 0.0594	0.3690 $\pm$ 0.0456
	S2	0.4394 $\pm$ 0.0127	0.3924 $\pm$ 0.0137	0.4885 $\pm$ 0.0382	0.4557 $\pm$ 0.0269	0.4584 $\pm$ 0.0104
	S3	0.4461 $\pm$ 0.0193	0.4420 $\pm$ 0.0406	0.4006 $\pm$ 0.0154	0.3958 $\pm$ 0.0799	0.4160 $\pm$ 0.0597
	S4	0.5909 $\pm$ 0.0564	0.5801 $\pm$ 0.0772	0.6071 $\pm$ 0.0701	0.5392 $\pm$ 0.0650	0.5751 $\pm$ 0.0683

Appendix

Table 10: Per-suite results on PaperBench. We report functionality and maintainability metrics across five disjoint suites of four tasks each, as mean

\pm

std over three independent trials per suite. Avg. is pooled across all 15 trials (5 suites

\times

3 trials). Library Size reports the size of the final Super Library only.

	Patch				Change in maintainability					Patch Size by suite
Method	Added LOC $\downarrow$	App $\downarrow$	Lib	Files $\downarrow$	$\Delta$ LOC $\downarrow$	$\Delta$ Tok $\downarrow$	$\Delta$ MDL $\downarrow$	$\Delta$ Eros. $\downarrow$	$\Delta$ Verb. $\downarrow$	S1	S2	S3
Zero-Shot	936	936	0	29.8	$+$ 605	$+$ 5101	$+$ 2565	$-$ 0.004	$+$ 0.007	182	1804	822
Librarian	522	500	22	27.0	–	–	–	–	–	151	717	699
Naive-Implicit	632	618	14	27.4	$+$ 278	$+$ 2874	$+$ 2078	$+$ 0.019	$+$ 0.005	97	1328	471
Naive-Ward	380	363	18	22.2	$+$ 251	$+$ 2779	$+$ 1909	$+$ 0.030	$+$ 0.011	103	574	463
SLA-Full	256	232	24	19.9	$+$ 171	$+$ 2073	$+$ 1377	$+$ 0.001	$+$ 0.003	79	422	268

Appendix

Table 11: Post-construction maintenance, full results. The table extends Table 2 with the number of files touched, changes in portfolio-level structural metrics, and Patch Size for each suite.

\Delta

columns report post-patch minus pre-patch values. They are unavailable for Librarian , whose pre-patch portfolio is a post-hoc refactoring of the Zero-Shot codebases rather than a round-structured run.

App ID	Action	Expected result
000015	Go to the Contact Us page. In the contact form, fill the Name and Message fields but leave Email empty (or whitespace only), then submit.	An inline error message appears, no success/confirmation state is shown, and the values already typed in Name and Message remain in the fields.
000015	On the same contact form, fill Name, a valid Email containing “@” and “.”, and Message, then submit.	An in-page confirmation appears on the Contact Us page without navigating away, containing the word “Received” or “Submitted” and including the submitted name or email.
000016	Go to the Contact page. In the contact form, enter a Name and Message but type an invalid email without “@” or “.” (e.g., “testmail”), then submit.	A clear inline error appears for the email field, no confirmation banner is shown, and the typed Name, Email, and Message values remain in the inputs.
000016	Correct the email to a valid address containing “@” and “.”, then submit the form.	An in-page confirmation banner appears on the Contact page without navigating away, containing “Received” or “Submitted” together with the submitted name or email.
000019	Go to the Contact page. Submit the contact form with the “How can we help?” field empty while Name and Email are filled.	An inline error message appears, no confirmation is shown, and all values typed so far, including Company if entered, are preserved in the form.
000019	Fill Name, a valid Email containing “@” and “.”, and the “How can we help?” field, then submit.	An in-page confirmation appears on the Contact page without navigating away, including “Received” or “Submitted” plus the submitted name or email.

Appendix

Table 12: Change-specific tests for Suite 1. Each row reproduces one WebVoyager action and its expected result.

App ID	Action	Expected result
000022	Create a new call log via the New Call form with required fields filled.	A confirmation message appears, and the new call log displays a human-readable ID with the CALL- prefix (e.g., CALL-001) and a “Last updated” timestamp. The call appears in the Open Calls or To-Do Calls view.
000022	Modify an existing client’s information (or close/mark done an existing call).	A confirmation message appears and the record’s “Last updated” timestamp refreshes to reflect the change.
000023	Create a new lead via the New Lead form/modal with valid details.	A confirmation message appears, and the new lead is shown with a LEAD- prefixed ID (e.g., LEAD-001) and a “Last updated” timestamp or date, visible in the leads table.
000023	Change that lead’s status (or edit it), then locate it using the leads search or filters.	A confirmation appears, the “Last updated” value refreshes, and the lead is findable via search/filter with its LEAD- ID still displayed.
000044	Register a new product via the registration form.	A confirmation message appears and the product’s SCT- prefixed ID is displayed immediately upon registration.
000044	Add a checkpoint to that product on the Track screen, then look it up on the Inquiry screen.	A confirmation appears for the checkpoint, the product’s “Last updated” line refreshes, and the product is findable via the Inquiry search with its updated status.

Appendix

Table 13: Change-specific tests for Suite 2. Each row reproduces one WebVoyager action and its expected result.

App ID	Action	Expected result
000027	Share a new promotion using the share form with all necessary details.	A confirmation message appears, and the new promotion is visible in the browse list carrying a visible “Pending review” badge or status text. It is not hidden.
000027	Search for that promotion using its title typed in mixed case with leading and trailing spaces, then search for a nonsense string such as “zzqqxx”.	The mixed-case padded query still returns the promotion. The nonsense query shows a clear empty-state message instead of a blank area or all deals.
000051	Post a new job as a client with all required details.	A confirmation appears, and the new listing shows a visible “Pending review” badge while still appearing on the jobs board and in the client’s active-postings list.
000051	Search the jobs page for that job’s title typed in mixed case with surrounding spaces, then search for a nonsense string.	The padded mixed-case query returns the job. The nonsense query shows a clear empty-state message instead of a blank list or all jobs.
000052	On the jobs page, search for “ELECTRICIAN ” (upper case with trailing spaces), then search for a nonsense string such as “zzqqxx”.	The padded upper-case query returns the electrician listings. The nonsense query shows a clear empty-state message such as “No jobs match your search” instead of a blank grid or all listings.
000053	As a company, post a new internship position.	A confirmation appears, and the new listing shows a visible “Pending review” badge while remaining visible on the student browse list and in the company’s listings.

Appendix

Table 14: Change-specific tests for Suite 3. Each row reproduces one WebVoyager action and its expected result.

Method	WebGen	PaperBench
Zero-Shot	1.0	1.0
Librarian ( $K{=}8$ )	5.9	3.2
Naive-Implicit	3.1	1.7
Naive-Ward	2.9	1.7
SLA-Full	8.3	3.1

Appendix

Table 15: Cost of producing one portfolio , as total LLM spend relative to Zero-Shot (

\times

ZS, lower is better); same runs as Table 1 . Librarian (

K{=}1

) is omitted: it reuses a sample already drawn for

K{=}8

and has no independent cost.

	Cost		Functionality		Maintainability
Migration configuration	Total ( $)$ \downarrow$	Migr. turns $\downarrow$	Acc. $\uparrow$	Appr. $\uparrow$	LOC $\downarrow$	MDL $\downarrow$	Eros. $\downarrow$	Verb. $\downarrow$
Fresh sessions	0.453	485	75.4	3.78	8598	34473	0.094	0.104
Reused sessions	0.400	354	76.0	3.88	8760	35122	0.104	0.114
$\Delta$ (%)	$-$ 11.7	$-$ 27.0	$+$ 0.8	$+$ 2.6	$+$ 1.9	$+$ 1.9	$+$ 11.1	$+$ 10.0
$p$	0.017	0.002	0.749	0.064	0.348	0.553	0.410	0.536

Appendix

Table 16: Reusing the dependency-migration session across rounds (WebGen-Bench). Mean over three suites

\times

three trials (

n{=}9

), both arms seeded from the same round-1 portfolio.

p

is from a two-sided paired

t

-test, with

p<0.05

in bold. Cost is the total spend to produce one portfolio, as in Table 1 . Reusing the session cuts cost and migration turns significantly while every functionality and maintainability metric is statistically indistinguishable.

$m$	Rounds	Cost $\downarrow$	Acc. $\uparrow$	LOC $\downarrow$	Verb. $\downarrow$
2	4	8.30	77.21	8552	0.099
4	2	6.55	73.97	8643	0.095
8	1	4.93	74.44	8932	0.110

Appendix

Table 17: Applications per round (WebGen-Bench). The portfolio is fixed at eight applications;

m

sets how many arrive per round, so larger

m

means fewer extract-and-migrate cycles.

m{=}2

is the configuration used everywhere else in the paper. Cost is relative to Zero-Shot, as in Table 15 ; mean over three suites

\times

three trials.

	Zero-Shot	Naive-I	Naive-W	SLA-Full
Functionality
Acc. $\uparrow$	79.17	74.51	81.68	77.45
Appr. $\uparrow$	3.92	3.86	3.90	3.91
Maintainability
LOC $\downarrow$	10,957	11,305	11,152	10,581
Tok $\downarrow$	80,796	85,795	85,930	80,185

Appendix

Table 18: Results with the Minimax-M3 backbone (WebGen-Bench). Mean over three suites

\times

three trials; bold marks the best method per row. Standard deviations are omitted for space and follow the same pattern as Table 1 .

	Zero-Shot	Naive-I	Naive-W	SLA-Full
Functionality
Acc. $\uparrow$	69.02	62.58	63.37	65.88
Appr. $\uparrow$	3.66	3.48	3.52	3.29
Maintainability
LOC $\downarrow$	23,263	20,744	19,900	18,557
Tok $\downarrow$	177,022	163,063	157,096	147,844

Appendix

Table 19: Stress test: the hardest suite of

N{=}16

tasks (WebGen-Bench). Mean over three trials; bold marks the best method per row. Metrics are computed with the same evaluation harness as the main results. Standard deviations are omitted for space.

		Functionality		Maintainability
Method	Scaffold	Acc. $\uparrow$	Appr. $\uparrow$	LOC $\downarrow$	Tok $\downarrow$	MDL $\downarrow$	Eros. $\downarrow$	Verb. $\downarrow$
Zero-Shot	mini-SWE-agent	76.04	3.84	9393	70364	34875	0.1032	0.1603
Zero-Shot	OpenHands	77.34	3.92	8948	68528	34926	0.0732	0.1489
Zero-Shot	Claude Code	78.20	3.86	8712	67857	34009	0.1125	0.1349
SLA-Full	mini-SWE-agent	77.21	3.86	8552	65633	34195	0.0987	0.0994

Appendix

Table 20: Scaffold comparison (WebGen-Bench). All rows use the same backbone ( deepseek-v4-flash ), task specifications, and reasoning-effort setting, and only the scaffold differs. Mean over three suites

\times

three trials.

Method	Trial	#	Exported symbols with reuse rate
Naive-Implicit	T1	6	Footer (100%), Header (88%), ContactForm (62%), PillNav (50%), useLocalStorage (50%), SkillBar (25%).
Naive-Implicit	T2	5	Footer (100%), Header (88%), ContactForm (62%), useLocalStorage (62%), SkillBar (25%).
Naive-Implicit	T3	6	Footer (100%), ContactForm (50%), NavBar (50%), FilterGroup (38%), useLocalStorage (38%), SkillBar (25%).
Naive-Ward	T1	4	Footer (100%), Header (62%), ContactSection (38%), SkillsSection (25%).
Naive-Ward	T2	5	Footer (100%), Header (88%), ContactForm (62%), SkillsChart (25%), useLocalStorage (25%).
Naive-Ward	T3	7	Footer (88%), Header (62%), ContactForm (50%), Section (38%), loadStorage (38%), saveStorage (38%), SkillBar (25%).

Appendix

Table 21: Complete Super Library export inventory for WebGen-Bench Suite 1. Reuse rate denotes the percentage of the eight applications in the suite that import each symbol.

Method	Trial	#	Exported symbols with reuse rate
Naive-Implicit	T1	10	Button (100%), FormField (88%), Card (75%), useLocalStorage (75%), ErrorMsg (62%), NavBar (50%), StatusPill (50%), SearchInput (38%), loadStorage (0%), saveStorage (0%).
Naive-Implicit	T2	10	useLocalStorage (100%), Button (88%), Card (88%), FormField (75%), KpiCard (75%), Footer (50%), NavBar (50%), SearchBar (38%), PageHeader (12%), DataTable (0%).
Naive-Implicit	T3	9	Footer (100%), Header (100%), NavBar (88%), Badge (62%), useLocalStorage (62%), Card (50%), loadStorage (38%), saveStorage (12%), StatCard (0%).
Naive-Ward	T1	7	Button (88%), Card (88%), useLocalStorage (62%), Footer (50%), Header (50%), NavBar (50%), Sidebar (25%).
Naive-Ward	T2	9	Button (100%), Card (88%), useLocalStorage (75%), FormField (62%), StatusPill (62%), StatCard (50%), Table (50%), NavTabs (38%), Alert (12%).
Naive-Ward	T3	14	Button (88%), useLocalStorage (75%), Card (62%), StatCard (62%), Badge (50%), FormGroup (50%), NavBar (50%), DataTable (38%), PageHeader (25%), SearchBar (25%), Sidebar (25%), EmptyState (12%), loadData (0%), saveData (0%).

Appendix

Table 22: Complete Super Library export inventory for WebGen-Bench Suite 2. Reuse rate denotes the percentage of the eight applications in the suite that import each symbol.

Method	Trial	#	Exported symbols with reuse rate
Naive-Implicit	T1	8	Alert (100%), loadStorage (100%), saveStorage (100%), Footer (62%), FileInput (38%), SearchBar (38%), useFileUpload (25%), useLocalStorage (0%).
Naive-Implicit	T2	12	Alert (100%), FormField (100%), useLocalStorage (100%), validateRequired (50%), FilePicker (38%), Button (25%), SearchBar (25%), validateEmail (25%), validateMinLength (25%), Modal (12%), Pill (12%), Footer (0%).
Naive-Implicit	T3	8	useFormValidation (75%), Footer (62%), createKeyedStorage (50%), loadStorage (50%), saveStorage (50%), Chip (38%), FileUpload (38%), SearchFilter (38%).
Naive-Ward	T1	5	Header (100%), loadStorage (100%), saveStorage (100%), Footer (75%), FormField (75%).
Naive-Ward	T2	8	Button (62%), Footer (62%), FormField (62%), createAuthContext (62%), Modal (25%), useLocalStorage (12%), getStorage (0%), setStorage (0%).
Naive-Ward	T3	6	loadJSON (100%), saveJSON (100%), Footer (62%), ImageUpload (38%), LoginForm (38%), RegisterForm (25%).

Appendix

Table 23: Complete Super Library export inventory for WebGen-Bench Suite 3. Reuse rate denotes the percentage of the eight applications in the suite that import each symbol.

Figure 7: All eight Content-Presentation tasks under trial 1. Each task is shown as a (without library, with library) pair; the eight tasks are tiled as four rows of two tasks. UI-test accuracy and appearance grade are reported below each pair.

Artifact	License	Use
WebGen-Bench	MIT	eval suite
PaperBench	MIT	eval suite
WebVoyager	Apache 2.0	UI runner
mini-SWE-agent	MIT	harness
cocoindex-code	Apache 2.0	indexing
Qwen2.5-7B	Apache 2.0	MDL ref.

Appendix

Table 24: External artifacts and their licenses. All are permissive open-source licenses, and our use is consistent with each artifact’s intended purpose.

Automated Code Refactoring

See all 16 papers

Jul 20, 2026

TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization

Coding agents are increasingly used to accelerate code generation in many downstream tasks, such as fixing bugs, building applications, and prototyping. However, despite their value as coding assistants, agent-generated code tends to be larger and more verbose than the corresponding human-written implementation. In this work, we show that the cause lies in the agent's own search process: while iterating toward a passing solution, an agent accumulates speculative edits, abandoned hypotheses, and temporary changes that persist into the final patch. This may seem harmless for a single patch, but the problem compounds as agents take responsibility for ever-larger portions of a codebase-a codebase that was once minimal and well-maintained slowly accumulates redundancy faster than it can be cleaned up, drifting to a state that is harder to maintain. Given the magnitude of this problem, we take a step towards alleviating this issue. First, we formally define this phenomenon as CodeSlop-the residual and functionally unnecessary edits commonly seen in AI-generated code. We then introduce our algorithm TRIM (Trajectory-guided Redundancy Identification and Minimization). Rather than minimizing CodeSlop directly, TRIM instead minimizes agent trajectories. As we show empirically, this indirect technique of minimizing CodeSlop is highly effective: TRIM cuts CodeSlop by 17.9%-32.9% across agentic scaffolds, with negligible performance regression. TRIM is also highly efficient, requiring roughly half the validation cost of algorithmic baselines such as Delta Debugging.

Alex Mathai, Shobini Iyer, Aleksandr Nogikh +4

Dept. of Computer Science Columbia University, New York City, USA · Google Inc München, Germany · Google DeepMind Mountain View, CA, USA +1

Coding Agents Agentic Code Generation

Ask PDF

Sep 29, 2026

E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch

Coding agents powered by large language models (LLMs) are evolving from making localized code changes to developing complete software repositories. However, evaluating repository-scale generation remains challenging: tasks must demand system-level reasoning while ensuring that all evaluated behaviors are precisely specified and independent of any particular implementation. We introduce E2E-SWE, a benchmark for evaluating whether coding agents can build complete, functional software repositories end to end. E2E-SWE contains 186 whole-repository generation tasks spanning 11 programming languages. Given only a natural-language specification and an empty workspace, an agent must implement a complete, installable project that satisfies a comprehensive suite of hidden tests. Each task is constructed by a software engineer in collaboration with an LLM; together, they develop the test suite and a corresponding implementation-independent specification. To ensure that tasks are well specified and practically solvable, we further subject them to an iterative verification process in which autonomous agents audit and repair task defects using static inspection and failures observed from real model rollouts. Evaluating 13 frontier models, we find substantial variation in end-to-end repository generation ability, with pass@1 ranging from 11.7% to 67.7%, providing strong model differentiation while leaving considerable headroom for future progress. Analysis of agent trajectories further reveals long, front-loaded reasoning patterns, highlighting the planning and system-level reasoning required to construct working codebases from scratch.

Hantian Ding, Chloe Bi, Jiacheng Zhu +5

Meta Superintelligence Labs

Software Engineering Agents Repository-Level Code Generation Code Generation Benchmarks

Ask PDF

Jul 1, 2026

Agent4cs: A Multi-agent System for Code Summarization in Large Hierarchical Codebases

Understanding large, complex codebases, especially those with obfuscated structures and incomplete documentation, remains a significant challenge. Existing code summarization solutions often rely on a single language model or coding assistant like Claude Code, and treat source code as flat text, underutilizing the rich interdependencies and hierarchical information within a repository. To address these shortcomings, we propose Agent4cs - a multi-agent framework that summarizes large codebases in a bottom-up fashion, where a summarization agent focuses on producing robust summaries; a keyword-extraction agent proactively identifies critical information from subfolders; and a quality-assurance agent iteratively refines the outputs for readability, coherence, and completeness. Evaluated on 7 frontier models, Agent4cs improves semantic consistency across all folder levels by average 8% compared to two structured prompting baselines with code segments. Furthermore, extensive evaluation on real-world datasets demonstrates up to 38% gains in normalized keyword coverage rate over the same baselines.

Yongjian Tang, Ezgi Sarikayak, Doruk Tuncel +2

Siemens AG · Technical University of Munich · Munich, Bayern, Germany +2

Multi-Agent LLM Systems Software Engineering Agents Text Summarization

Ask PDF

Abstract

Automated Code Refactoring

Explore similar work

TRIM: Reducing AI-Generated CodeSlop via Agent Trajectory Minimization

E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch

Agent4cs: A Multi-agent System for Code Summarization in Large Hierarchical Codebases