Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training

Tingyun Li; Wenfeng Feng; Weiqing Li; Abudukelimu Wuerkaixi; Guohua Liu; Yuewei Zhang

cs.AIAug 27, 2026

Knowing When Not to Reuse: Conditional Experience Transfer in Autonomous LLM Post-Training

Authors: Tingyun Li, Wenfeng Feng, Weiqing Li, Abudukelimu Wuerkaixi, Guohua Liu, Yuewei Zhang

Organizations: Alibaba Cloud Computing

Abstract

Large language models offer broad capabilities, but adapting them to evolving domains, tools, and requirements often entails repeated post-training. Autonomous systems automate parts of this process by proposing updates, training candidates, and using evaluation feedback to select subsequent proposals. As evidence accumulates, a central problem emerges: which past update evidence remains actionable after subsequent training has changed the parent model? An update's effect depends on its parent, data, and training stage. Treating past success as context-free permission can waste compute. If the resulting child is promoted, it can also degrade the subsequent training trajectory. We formulate this problem as conditional experience transfer and introduce Boundary-Calibrated Intervention Transfer (BCIT), a method that authorizes experience reuse before weight-changing training. BCIT binds an observed effect to its source context, checks applicability conditions, vetoes candidates with named hard conflicts, and obtains current-state evidence through a bounded training trial when needed. Fully trained candidates still face a shared adoption rule, and only observed events extend memory. On one 4B model adapted across finance reasoning, text-to-SQL, and function calling, candidate updates exhibit heterogeneous target and retention effects across the evaluated contexts. Under matched candidates, evidence, and compute, BCIT authorizes fewer harmful updates and attains higher equal-budget final-model quality than the evaluated alternatives. These results support treating experience authorization as a distinct problem in autonomous post-training.

Figures & tables

Figure 1: BCIT uses source-context evidence to authorize candidates for full training. Historical records combine source strength, current compatibility, and a hard-conflict veto. A new proposal has no source score: it enters validation only if it passes the shared executability check, has no hard conflict, and admits a faithful short run; otherwise it is rejected. A validation checkpoint is never promoted. Every fully trained child faces the same adoption rule, and only observed events extend memory.

Figure 2: Candidate updates exhibit heterogeneous target and retention effects across 24 matched candidate–context pairs. (a) Target-direction counts (

n=8

per family); arrows redundantly encode direction. (b) Target change versus the worse IFEval change. The gray

[-2,2]

band is linear; the disclosed symmetric-log scale compresses only larger retention changes. Semantic labels replace run identifiers.

Policy	Authorized B/H/N	All auth.	Harmful auth. $\downarrow$	Beneficial kept $\uparrow$	Harmful share $\downarrow$
Reuse all	10/8/6	100.0	100.0	100.0	33.3
Source evidence only	9/6/4	79.2	75.0	90.0	31.6
Applicability only	8/5/4	70.8	62.5	80.0	29.4
Flat-Additive	8/5/3	66.7	62.5	80.0	31.3
BCIT	9/2/3	58.3	25.0	90.0	14.3
BCIT w/o candidate-specific applicability	9/6/3	75.0	75.0	90.0	33.3

Table 1: Outcome-blind Audit-24. Policies receive the same frozen candidate records and current-context metadata, fix decisions before full outcomes open, and are scored against the same 10 beneficial, 8 harmful, and 6 neutral outcomes. Rates are percentages.

Method	TAT	BIRD	BFCL	Mean	Min. $\Delta$	IFEval P/I	GPUh	AUC
Base	31.3	39.0	57.2	42.5	0.0	76.5/82.7	0.0	42.5
Fin. specialist	39.6	–	–	–	–	–	36.0	–
SQL specialist	–	49.3	–	–	–	–	36.0	–
Func. specialist	–	–	65.1	–	–	–	36.0	–
Flat-Add.	33.4	41.2	58.5	$44.4\pm 0.3$	+1.3	75.0/80.9	$35.9\pm 0.1$	43.6
Validate-All	34.6	42.5	59.3	$45.5\pm 0.2$	+2.1	75.6/81.7	$35.8\pm 0.1$	44.2

Table 2: 36-GPU-hour endpoint results. TAT: TAT-QA; P/I: prompt-/instruction-level IFEval. Cross-task mean and GPUh are mean

\pm

sample SD; other entries are means. Values are rounded to one decimal; tests use unrounded seed-level data. Bold marks the best shared-model value in each performance column.

Figure 3: Observed equal-budget trajectories. Mean task score versus consumed budget; BCIT and Flat-Additive bands are

\pm 1

sample SD over six paired seeds.

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Study	Scientific role	Outcome access and sampling unit	Material reported here
Retrospective-24	Diagnose effect heterogeneity	Outcomes exposed before analysis; 24 matched candidate–context pairs	All target and retention deltas; semantic intervention labels
Audit-24	Compare authorization decisions from matched information	Decisions frozen before full outcomes open; 24 decision units	Policy counts, structural ablations, exact intervals, and paired discordance table
ShortFull-24	Measure bounded-validation fidelity	Every short/full pair executed; no pass-only filtering; 24 candidates	Full short/full ledger, sign reversals, confusion counts, and budget sensitivity
36-GPU-hour episodes	Compare complete sequential policies under equal compute	Six matched episodes for the primary and protocol-amendment comparisons	Seed-level endpoints, paired intervals, component directions, process counts, and aggregate endpoints

Appendix

Table 3: Relationship between the four empirical components. Unit rows are observed candidate-level or episode-level values rather than reconstructed pseudo-observations.

Method	Historical library	New proposals	Pre-full-training policy	Candidate stream and adoption
BCIT	frozen	at most 3	product, hard veto, selective validation	matched stream; shared $g$
Flat-Additive	same as BCIT	same as BCIT	additive no-veto score; otherwise matched	matched stream; shared $g$
Validate-All	same candidates	same as BCIT	validate every executable candidate	matched stream; shared $g$
Additive+Veto	same as BCIT	same as BCIT	additive score with hard veto and BCIT validation	matched stream; shared $g$
BCIT-Retrieve	frozen	disabled	BCIT authorization	retrieved library candidates only; shared $g$
No-memory search	disabled	target-only	every new proposal requires current-parent validation	method-specific proposal stream; shared $g$

Appendix

Table 4: Operational contracts for the shared-model methods in the endpoint table. The first four policies share seed-indexed candidate streams and adoption rule

g

; the remaining rows are descriptive baselines with the stated action spaces.

Illustrative update	$S_{i}$	$A_{i}$	$H_{i}$	$\chi_{i}$	Decision	Reason
Rationale-first SQL SFT in a matched execution setting	1.00	.80	0	1	Train	$Q_{i}=.80\geq\tau_{h}$ and replicated source evidence permits direct training
Low-rate finance continuation with single-run evidence	.45	.90	0	0	Validate	$Q_{i}=.405$ is not rejected, but grade B evidence cannot bypass current-state validation
Function-calling update under an incompatible output schema	.90	.80	1	1	Reject	The named protocol conflict is non-compensable even though the positive score is high

Appendix

Table 5: Constructed authorization cases illustrating the frozen rule. These examples explain routing only; they are not experimental observations and do not contribute to any reported result.

Family	Source	Target	Metric / count
Finance	FinQA	TAT-QA	Official F1 / 1,663
Text-to-SQL	Spider	BIRD	Exec. acc. / 400
Function calling	xLAM	BFCL	Full match / 444

Appendix

Table 6: Capability families and frozen target-evaluation cohorts. Counts are evaluation examples, not training-set sizes.

Field	Value
Base model	Qwen3-4B
Framework	LlamaFactory / PyTorch
Precision	bfloat16
Distributed optimizer	DeepSpeed ZeRO-2
Context length	4,096 tokens
Learning rate	$6\times 10^{-6}$

Appendix

Table 7: Common full-parameter SFT defaults.

Field	Frozen value
Semantic update	Rationale-first SQL SFT
Atomic change	Add 260 train-only SQL rationale examples to the base-plus-retention SFT recipe
Required conditions	Executable SQLite databases and schema context; execution accuracy as the primary metric; compatible parent family and training stage
Named conflicts	Dialect/backend mismatch; incompatible output schema; missing required database access
Provenance	Training data, configuration, evaluator, source outcome, and run identifier each carry a content digest
Decision-time visibility	Source record, evidence grade, applicability fields, and provenance; no target full-run metric or held-out target error case

Appendix

Table 8: Example frozen historical update record. This schema/provenance example is not an additional experimental outcome.

Family	Semantic update	Target $\Delta$	P $\Delta$	I $\Delta$	Family	Semantic update	Target $\Delta$	P $\Delta$	I $\Delta$
Fin.	Balanced numeric repair	$-.487$	$+1.109$	$+2.038$	SQL	Complex-query replay	$+.750$	$.000$	$-.120$
Fin.	Strict-CoT repair	$-.890$	$+8.133$	$+7.914$	SQL	Execution-reward GRPO	$+.750$	$+1.109$	$+.360$
Fin.	CoT cold start	$+.355$	$-.739$	$-.719$	SQL	Mixed execution GRPO	$-.750$	$.000$	$-.600$
Fin.	Exact-reward GRPO	$-.182$	$+1.479$	$+.839$	SQL	Hard-query GRPO	$.000$	$+.924$	$+.360$
Fin.	Extended SFT	$+.357$	$+.370$	$+1.679$	SQL	Rationale-first SFT	$-2.250$	$-1.109$	$-1.559$
Fin.	Precision replay	$+.352$	$+13.309$	$+11.990$	SQL	Rejection distillation	$-.250$	$+2.773$	$+2.398$

Appendix

Table 9: All Retrospective-24 matched effects in percentage points. Each row is intervention minus its matched control. P/I denote IFEval prompt/instruction strict accuracy.

Policy	Auth. B/H/N	Coverage	HER	BCov	Harm/Auth.	$\mathrm{SelU}_{2}$
Reuse all	10/8/6	100.0%	100.0%	100.0%	33.3%	$-.250$
Source only	9/6/4	79.2%	75.0%	90.0%	31.6%	$-.125$
Applicability only	8/5/4	70.8%	62.5%	80.0%	29.4%	$-.083$
Flat-Additive	8/5/3	66.7%	62.5%	80.0%	31.3%	$-.083$
BCIT	9/2/3	58.3%	25.0%	90.0%	14.3%	$\mathbf{+.208}$
BCIT w/o candidate-specific applicability	9/6/3	75.0%	75.0%	90.0%	33.3%	$-.125$

Appendix

Table 10: Audit-24 authorization results and structural ablations. Lower HER and Harm/Auth. are better; higher BCov is better.

Metric	BCIT	Flat-Additive
Coverage	$[36.6,77.9]$	$[44.7,84.4]$
HER	$[3.2,65.1]$	$[24.5,91.5]$
BCov	$[55.5,99.7]$	$[44.4,97.5]$
Harm/Auth.	$[1.8,42.8]$	$[11.0,58.7]$

Appendix

Table 11: Audit-24 95% marginal exact-binomial intervals.

Figure 4: Candidate-level bounded-validation fidelity. Circles and triangles denote SFT and GRPO; open rings mark the four sign reversals.

ID	Type	Label	Short $\Delta$	Full $\Delta$	Action	ID	Type	Label	Short $\Delta$	Full $\Delta$	Action
S01	SFT	H	$-.13974$	$-.100$	Fail	G01	GRPO	H	$-.21813$	$-.126$	Fail
S02	SFT	H	$-.14760$	$-.072$	Fail	G02	GRPO	H	$-.16441$	$-.083$	Fail
S03	SFT	H	$-.12255$	$-.051$	Fail	G03	GRPO	H	$-.10990$	$-.047$	Fail
S04	SFT	H	$+.15270$	$-.030$	Pass	G04	GRPO	H	$+.09411$	$-.023$	Pass
S05	SFT	N	$-.06664$	$-.012$	Fail	G05	GRPO	N	$-.22667$	$-.010$	Fail
S06	SFT	B	$+.05525$	$+.012$	Pass	G06	GRPO	N	$-.09494$	$-.003$	Fail

Appendix

Table 12: Candidate-level ShortFull-24 ledger at the primary 20% validation budget. Short and full target changes are raw score fractions.

Budget	P/F/I	Sign agree	$\rho$	Median cost/full
5%	9/7/8	62.5%	.42	.06
10%	10/8/6	75.0%	.61	.11
20%	10/9/5	83.3%	.72	.17

Appendix

Table 13: Short-budget sensitivity on the same 24 candidates. P/F/I denote Pass/Fail/Inconclusive counts.

Method	Seed	TAT-QA	BIRD	BFCL	Mean	IFEval-P	IFEval-I	GPUh	Pair $\Delta$
BCIT	S1	35.400	44.000	60.586	46.662	76.895	83.094	34.7	$+2.559$
BCIT	S2	36.500	44.500	61.261	47.420	76.710	82.854	35.3	$+2.726$
BCIT	S3	35.800	43.750	60.811	46.787	76.710	82.974	35.0	$+2.351$
BCIT	S4	36.100	44.500	60.811	47.137	76.525	82.614	34.8	$+3.134$
BCIT	S5	35.100	44.000	60.360	46.487	76.895	83.094	35.6	$+1.834$
BCIT	S6	36.400	44.750	61.261	47.470	76.710	82.854	34.9	$+3.176$

Appendix

Table 14: Seed-level BCIT and Flat-Additive endpoints. Mean is recomputed from the three displayed target scores. Pair

\Delta

is BCIT mean minus the matched Flat-Additive mean.

Policy	Seed	TAT-QA	BIRD	BFCL	Mean	IFEval-P	IFEval-I	GPUh	AUC	$\Delta$
Validate-All	S1	34.720	42.600	59.350	45.557	75.620	81.740	35.74	44.280	$+1.105$
	S2	34.300	42.180	59.110	45.197	75.310	81.420	35.91	44.050	$+2.223$
	S3	34.880	42.740	59.440	45.687	75.800	81.910	35.62	44.370	$+1.100$
	S4	34.510	42.350	59.280	45.380	75.440	81.580	35.96	44.130	$+1.757$
	S5	35.020	42.810	59.520	45.783	75.930	82.030	35.68	44.420	$+.704$
	S6	34.460	42.490	59.200	45.383	75.550	81.690	35.83	44.190	$+2.087$

Appendix

Table 15: Seed-level endpoints for Validate-All and Additive+Veto.

\Delta

is the matched BCIT mean minus the baseline mean.

Policy	Seed	TAT-QA	BIRD	BFCL	Mean	IFEval-P	IFEval-I	GPUh	AUC	$\Delta$
BCIT w/o hard veto	S1	35.000	42.650	59.600	45.750	75.420	81.520	35.46	44.230	$+.912$
	S2	34.400	42.150	59.000	45.183	74.960	81.080	35.71	43.960	$+2.237$
	S3	34.700	42.420	59.300	45.473	75.210	81.310	35.58	44.100	$+1.314$
BCIT-Reject-Unresolved	S1	34.700	42.500	59.000	45.400	76.220	82.300	32.84	44.030	$+1.262$
	S2	34.150	42.050	58.700	44.967	75.910	82.020	33.12	43.790	$+2.453$
	S3	34.450	42.280	58.900	45.210	76.080	82.180	32.96	43.910	$+1.577$

Appendix

Table 16: Three-seed component ablations.

\Delta

is the matched BCIT mean minus the ablation mean on S1–S3. These rows are descriptive and receive no significance test.

Method	TAT-QA	BIRD	BFCL	Mean	Worst $\Delta$	IFEval-P	IFEval-I	GPUh	AUC
Base	31.3	39.0	57.2	42.5	0.0	76.5	82.7	0.0	42.5
Single-task specialists (36 GPU-hours each)
Finance specialist	39.6	–	–	–	–	–	–	36.0	–
SQL specialist	–	49.3	–	–	–	–	–	36.0	–
Function specialist	–	–	65.1	–	–	–	–	36.0	–
Shared-model baselines

Appendix

Table 17: All 36-GPU-hour endpoints. Task and retention columns are means; cross-task Mean and GPUh include sample SD. Secondary baselines are descriptive aggregates and are not used for inferential claims.

LLM Post-Training

Large language model post-training adapts pretrained models into more useful, reliable, and specialized assistants through supervised and preference-based learning.

See all 249 papers

What is Missing from AI Post-Training AI: An Empirical Analysis
Aug 19, 2026Joy Jia Yin Lim, Xin Huang, Hao Peng +5AI Agent Evaluation LLM Agent Self-Improvement
Tsinghua University · University of Electronic Science and Technology of China · Renmin University of China
Rethinking Continual Experience Internalization for Self-Evolving LLM Agents
Jun 3, 2026Jingwen Chen, Wenkai Yang, Shengda Fan +7Continual Learning for LLM Agents Continual Learning for LLMs
1Gaoling School of Artificial Intelligence, Renmin University of China · School of Software, Beihang University · 3Meituan
Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning
Apr 9, 2026Shiwan Zhao, Zhihu Wang, Xuyang Zhao +10LLM Post-Training RL for Language Models
Nankai University, China · Huawei Technologies Ltd., China

Aug 19, 2026J/K move · Enter open · S save

What is Missing from AI Post-Training AI: An Empirical Analysis

Joy Jia Yin Lim, Xin Huang, Hao Peng, Yaxi Lu+4

Tsinghua University · University of Electronic Science and Technology of China · Renmin University of China

AI Agent Evaluation LLM Agent Self-Improvement LLM Agents LLM Post-Training Recursive Self-Improvement

Open paper AskPDF arXiv

Large language model (LLM) agents can now post-train an LLM end-to-end, raising the prospect of recursive self-improvement (RSI). Yet this progress is measured by aggregate benchmark scores, which cannot tell whether an agent executes a fixed plan well or strategically revises the plan when it fails. We separate these two capabilities: execution-level capability, iterating within an established training strategy, and strategy-level capability, revising that strategy as experimental evidence accumulates. Analyzing 1,338 post-training trajectories of frontier agents, we find that agents reliably execute post-training but lock into a default strategy, which follows the agent rather than the task, and only 2.1% of transitions between adjacent training runs ever change strategy. We then test whether the agent lacks experience, reasoning, or the decision to switch. (1) Experience improves execution but not the strategy. (2) Additional reasoning compute yields front-loaded gains on easier tasks but refines, rather than revises, the committed strategy. (3) Human review before training changes which strategy the agent locks into, not whether it locks in, whereas a single mid-run instruction outperforms the agent's own continuation by up to 17.44 points under the same budget. In conclusion, what the agent lacks is the decision to reopen a committed strategy and try another one. Realizing RSI therefore calls for interaction protocols and training signals that make strategy revision an explicit, rewarded decision.

Abstract

LLM Post-Training

Explore similar work

What is Missing from AI Post-Training AI: An Empirical Analysis

Rethinking Continual Experience Internalization for Self-Evolving LLM Agents

Large Language Model Post-Training: A Unified View of Off-Policy and On-Policy Learning