PAPER2LLM++: Continual Self-Evolution of LLMs from Research Papers

Hongji Pu; Yilun Zhao; Wenpeng Yin

cs.AIOct 2, 2026

PAPER2LLM++: Continual Self-Evolution of LLMs from Research Papers

Authors: Hongji Pu, Yilun Zhao, Wenpeng Yin

Organizations: UIUC · Yale University · Penn State

Abstract

Research on LLMs continually uncovers model limitations, their causes, and potential solutions. Yet these human discoveries remain largely disconnected from model evolution: an LLM does not automatically learn from new research about its own failures. We introduce PAPER2LLM++, a framework for continual self-evolution of LLMs from research papers. Rather than treating papers merely as knowledge to retrieve, PAPER2LLM++ uses the growing literature as a stream of evidence and supervision for model improvement. For each incoming paper, it extracts evidence-grounded findings, tests whether the reported limitation persists in the current model, and, when needed, converts the findings into candidate learning signals. A try-evaluate-commit procedure integrates an update only when it improves the targeted behavior without substantially forgetting prior improvements or degrading general capabilities. Across a sequential stream of research-discovered LLM failures, we show that models can progressively incorporate new findings while retaining earlier gains. PAPER2LLM++ thus takes a step toward closing the loop between human discovery and model evolution, enabling models to continually learn from research about their own limitations and improvements.

Figures & tables

Figure 1: Paper2LLM++ overview. (1) Paper evidence: extract grounded findings and verify the reported gap. (2) Data generation: translate confirmed gaps into complementary synthetic supervision. (3) Controlled adaptation: select LoRA updates based on target gains, retention, and general capabilities.

Paper Family	#papers	All Papers / Benchmarks	Typical Failure Pattern
Mathematical robustness	6	GSM-Symbolic ( Mirzadeh et al., 2025 ) , GSM-Plus ( Li et al., 2024a ) , MATH-Perturb ( Huang et al., 2025 ) , MathCheck ( Zhou et al., 2025 ) , Math-RoB ( Yu et al., 2025 ) , MathVista ( Lu et al., 2024 )	rules, constraints, verification
Coding and software	6	HumanEval+ ( Liu et al., 2023 ) , LiveCodeBench ( Jain et al., 2025 ) , CRUXEval ( Gu et al., 2024 ) , EvoEval ( Xia et al., 2024 ) , SWE-bench ( Jimenez et al., 2024 ) , CrossCodeEval ( Ding et al., 2023 )	execution, retrieval, interaction
Instruction operators	4	Negation ( Jang et al., 2022 ) , Reversal Curse ( Berglund et al., 2024 ) , Instruction Hierarchy ( Wallace et al., 2024 ) , Prompt Injection ( Li et al., 2024b )	boundaries and control policies
Knowledge and factuality	4	TruthfulQA ( Lin et al., 2022 ) , SimpleQA ( Wei et al., 2024 ) , GPQA ( Rein et al., 2024 ) , MMLU-Pro ( Wang et al., 2024 )	retrieval and calibrated reasoning
Context grounding	3	Lost-in-the-Middle ( Liu et al., 2024a ) , RULER ( Hsieh et al., 2024 ) , RGB ( Chen et al., 2024 )	retrieval, memory, trust
Structured reasoning	3	PlanBench ( Valmeekam et al., 2023b ) , Self-Critique Plans ( Valmeekam et al., 2023a ) , GraphQA ( He et al., 2024 )	planning and state verification

Table 1: Evaluation suite of 30 failure-revealing papers and benchmarks for paper-driven LLM self-evolution.

Figure 2: Results of Experiment I: individual-paper self-evolution. (a) Performance gains vary widely even within the same failure family. (b) 24 of 30 papers show positive held-out gains, while the remaining failures largely involve retrieval, planning, interaction, or verification capabilities beyond the current self-evolution framework.

Base LLM: Llama-3.1-8B-Instruct ( Grattafiori et al., 2024 )
Method	Same-Family Streams			Mixed 30-Paper Stream
Method	Gain $\uparrow$	Ret. $\uparrow$	Forget $\downarrow$	Gain $\uparrow$	Ret. $\uparrow$	Forget $\downarrow$	Gen. $\Delta$ $\uparrow$
Text-To-LoRA ( Charakorn et al., 2025 )	$+5.8_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 1.0}}$	$64.9\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 5.1}}$	$4.2_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.7}}$	$+5.1_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.9}}$	$62.6\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 4.3}}$	$5.6_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.8}}$	$-2.0_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.5}}$
ELDER ( Li et al., 2025a )	$-0.7_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.6}}$	$79.8\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 3.1}}$	$2.4_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.5}}$	$+3.3_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.9}}$	$72.4\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 4.6}}$	$3.8_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.6}}$	$-1.1_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.3}}$
RLEdit ( Li et al., 2025b )	$\bm{+7.4}_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.9}}$	$73.5\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 4.4}}$	$3.1_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.5}}$	$\bm{+7.9}_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 1.1}}$	$65.2\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 5.0}}$	$4.9_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.7}}$	$-1.6_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.4}}$
UltraEdit ( Gu et al., 2026 )	$+4.3_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.7}}$	$78.8\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 2.9}}$	$2.7_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.6}}$	$+5.0_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.6}}$	$74.6\%_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 3.2}}$	$3.5_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.5}}$	$-1.0_{{\color[rgb]{0.45,0.45,0.45}\scriptstyle\pm 0.3}}$

Table 2: Continual self-evolution within (left) and across (right) paper families.

Figure 3: Understanding papers’ effective signals for LLM self-evolution: (a) Qualitatively: gains vary across knowledge types. (b) Quantitatively: supervision closure is associated with held-out gain (

r=0.87

; Appendix A.4 ).

Figure 4: Gain vs. {semant. novelty, behav. novelty}

Figure 5: Marginal gains decrease but remain positive throughout the 30-paper stream.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Temporary	Full
Precision	bfloat16	bfloat16
Quantization	none	none
LoRA rank / alpha	4 / 8	16 / 32
LoRA dropout	0.05	0.05
Learning rate	$10^{-4}$	$10^{-4}$
Effective batch size	64	64

Appendix

Table 3: Current training configuration used by the final pipeline. Sequence lengths are 2,048 for short-form tasks, 4,096 for code and structured tasks, and 8,192 for long-context tasks.

Method	Online work / paper	Current implementation
Paper2LLM++	$\leq 4\times 60+1500=1740$ steps	Up to four temporary candidates, then one 1,500-step full update; 128 new + 64 replay examples.
Seq-LoRA	1,500 steps	Same 128+64 examples; one full update without probing or rollback.
ELDER	$32\times 50=1600$ iterations	32 batches; 50 iterations each; batch 4, rank 8, four LoRA experts, top-2 routing.
UltraEdit	0 gradient steps	Closed-form update over 128 edits.
StableEdit	0 gradient steps	Closed-form update over 128 edits.
Text-to-LoRA	0 gradient steps	One hypernetwork forward from the standardized paper description.

Appendix

Table 4: Per-paper online optimization accounting. The shared paper stream is not compute matched: Paper2LLM++ can spend additional optimization on candidate probing, whereas several editing baselines perform no per-paper gradient optimization.

Method	Adaptation to the paper stream
Text-to-LoRA	Render $\mathcal{R}_{t}$ as the task description; generate one adapter per paper and merge in arrival order. The Qwen2.5-7B hypernetwork is trained on disjoint task descriptions/adapters and frozen before evaluation.
ELDER	Convert $K_{t}$ into edits; one paper is one continual stage; carry mixture-of-LoRA state, router, guided loss, and deferral across papers.
RLEdit	Pre-train and freeze the editor on the authors’ original editing data, then apply it sequentially to paper-derived edit sets.
UltraEdit	Replace the edit stream with $K_{t}$ ; retain the released editor, sequential loop, layer choices, and step size.
StableEdit	Use the same paper-to-edit conversion as UltraEdit and preserve lifelong-normalization statistics across papers.
Seq-LoRA	Use the same LoRA configuration, 128 new examples, 64 replay examples, and 1,500-step full-update budget as Paper2LLM++ , but update after every paper without failure detection, candidate probing/selection, or commit/rollback.

Appendix

Table 5: Adaptation of continual-learning and lifelong-editing baselines to the common paper stream. The interface standardizes paper-derived input exposure while preserving each method’s native update mechanism; it does not equalize optimization compute.

Language Model Self-Improvement

Language model self-improvement studies how models use their own outputs, feedback, and experience to improve with less external supervision.

See all 174 papers

Chain-of-Experience for Continual LLM Improvement
Aug 18, 2026Haoqin Tu, Yunhao Fang, Yizhong Wang +2Language Model Self-Improvement Continual Learning for LLMs
UC Santa Cruz · Bytedance Seed
When Is Enough Enough in Self-Evolving LLM Systems?
Oct 3, 2026Enoch Yin, Bin Liu, Zhengling QiEarly Stopping Language Model Self-Improvement
Department of Statistics, The George Washington University · Department of Statistics and Data Science, School of Management, Fudan University · Department of Decision Sciences, School of Business, The George Washington University
When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain
Sep 29, 2026Chenxu Wang, Chaozhuo Li, Xinze Shi +6Language Model Self-Improvement LLM Training
Beijing University of Posts and Telecommunications · Beijing Academy of Artificial Intelligence · Central South University +2

Aug 18, 2026J/K move · Enter open · S save

Chain-of-Experience for Continual LLM Improvement

Haoqin Tu, Yunhao Fang, Yizhong Wang, Cihang Xie+1

UC Santa Cruz · Bytedance Seed

Language Model Self-Improvement Continual Learning for LLMs LLM Evaluation

Open paper AskPDF arXiv

Humans continuously learn from experience, whereas conventional large language model (LLM) evaluations ignore the models' ability to improve through inference-time interaction. In this paper, we study how LLMs learn from iterative experience at test time, a setting we refer to as Chain-of-Experience (CoE), where models accumulate experiential traces through iterative interactions with self or environmental feedback to form a continual improvement loop beyond zero-shot inference. We instantiate CoE with diverse feedback mechanisms, including model self-feedback and environmental signals such as correctness or public coding test pass rates, and evaluate across math, coding, and knowledge domains using 8 LLMs, including GPT-5, Gemini-2.5 Pro, Claude-4.5 Sonnet. Our study shows that leveraging iterative experience consistently outperforms feedback-free baselines, achieving substantial gains with self feedback alone, alongside a 5.6% overall improvement and 19% lower API cost across tasks and models. We further show that combining complementary feedback channels (e.g., model and correctness signals) yields additional gains, and that CoE delivers higher accuracy per token than existing test-time strategies. We observe a positive correlation between LLM base ability and improvement capacity, and show that models remain robust under weak or spurious feedback, with different feedback contributing to distinct improvement aspects and most gains emerging early in the iterations.

Abstract

Language Model Self-Improvement

Explore similar work

Chain-of-Experience for Continual LLM Improvement

When Is Enough Enough in Self-Evolving LLM Systems?

When Updating Stops Being Learning: Rethinking LLM Self-Evolution via learnable information gain