cs.LGJun 18, 2026

Marginal Advantage Accumulation for Memory-Driven Agent Self-Evolution

Authors: Mingyu Yang, Keye Zheng, Congchao Cheng, Yujie Liu, Xingkang Lu, Fan Jiang, Yefei Zheng

Organizations: Alibaba International Digital Commerce Group

Abstract

In batch-style trace distillation, the same memory operation may receive contradictory feedback across different batches. Existing methods lack a cross-batch, operation-level evidence accumulation mechanism, making it impossible to distinguish stably effective operations from accidental hits. This paper formalizes the requirement as two structural conditions, alignability and comparability, and proposes Marginal Advantage Accumulation (MAA). MAA constructs differential signals to make them comparable across batches, accumulates signed evidence per operation via EMA, and ensures cross-batch traceability through semantic identity merging. As a post-processing architecture, MAA achieves the best results in 14 out of 16 settings across 4 benchmarks and 4 target models, consistently outperforming existing batch-level distillation baselines and matching or surpassing online alternatives in most settings, while reducing optimization-phase token consumption by approximately 75%.

Explore similar work

CardsList
  1. Trust-Region Behavior Blending for On-Policy Distillation

    May 29, 2026Daniil Plyusov, Alexey Gorbatovski, Alexey Malakhov +4On-Policy DistillationTrust Region