A Cross-Scenario Survey and Analysis of Self-Evolving Multimodal Agents
Abstract
Recent work shows that multimodal agents increasingly self-improve without recurring human annotation. However, each self-improvement mechanism is designed and evaluated within a single scenario, leaving its preconditions unverified under cross-scenario transfer. This survey investigates the structural source of asymmetric mechanism transfer across four scenarios: multimodal reasoning, multimodal generation, GUI and Web navigation, and embodied manipulation. We first formalize the self-evolving multimodal agent by four criteria and a differentia of self-reference, and adopt the three-stage self-evolving loop of experience generation, feedback acquisition, and update consolidation as our analytical framework to position the scenarios along ceilings on rollout count and feedback fidelity. Comparing self-evolving with fixed loops within each scenario separates the limitations that the scenario imposes from those inherent in the self-evolving design. We then recast each mechanism as a trade-off that raises a scarce factor by presupposing a task-structural resource, such as output equivalence or checkable outcomes, and judge it portable, conditional, blocked, or redundant across scenarios. We attribute the asymmetry to divergent combinations of scarce factors and resources across opposite transfer directions, and identify the matched-budget measurements that could overturn these still untested verdicts.
Figures & tables
| Survey | Scenarios | Loop decomposition | Focus |
| ✜ Multimodal agents | |||
| Durante et al. [ 1 ] arXiv’24 | EM, games, healthcare, video, NLP | – | Agent AI paradigm and applications |
| Xie et al. [ 2 ] arXiv’24 | MG, GW, EM, games, driving, video | – | Components and planner types |
| Hu et al. [ 26 ] ACL’25 | GW | – | Construction and evaluation of OS agents |
| Yao et al. [ 3 ] arXiv’25 | GW, EM, deep research, healthcare, driving, recommendation | – | Agentic capabilities of MLLMs |
| Huang et al. [ 27 ] arXiv’26 | MG | – | Agentic control of visual generation |
| Genus criteria | Differentia | ||||||
| Paradigm | C1 | C2 | C3 | C4 | SR | Defining Mechanism | Representative Works |
| ✜ Outside the genus | |||||||
| Self-Training † | ✓ | ✓ | ✓ | ✓ | Iterative training of a classifier on its own pseudo-labels over a static unlabeled pool. | Noisy Student [ 72 ] ; FixMatch [ 73 ] ; Meta Pseudo Labels [ 74 ] | |
| Test-Time Adaptation † | ✓ | ✓ | ( ) | Inference-time adaptation of a classifier to an external test stream under a self-supervised or entropy objective. | TTT [ 75 ] ; TENT [ 76 ] ; MEMO [ 77 ] ; CoTTA [ 78 ] ; EATA [ 79 ] | ||
| Test-Time Scaling | ✓ | ✓ | ✗ | – | ( ) | Additional inference compute per query by repeated sampling, voting, or search, with nothing persisting past the query. | Self-Consistency [ 80 ] ; Snell et al. [ 81 ] ; Corvid [ 82 ] |
| Self-Correction | ✓ | ✓ | ✗ | – | ( ) | Refinement of an output through critique within one episode. | Self-Refine [ 83 ] ; CRITIC [ 84 ] ; Self-Debug [ 85 ] ; CoVe [ 86 ] |
| MR | MG | GW | EM | |
| ① Quantity factor : how many rollouts can a cycle afford? | ||||
| ✜Cost of one rollout | one decoding pass | one sampling pass or tool chain | per-step environment latency | per-step latency plus episode reset |
| ✜Action recoverability | none needed (no side effects) | none needed (no side effects) | partial (DOM and account-level writes) | largely irreversible (contact, wear) |
| ✜Reported wall-clock cost | not reported per rollout (one pass) | not reported per rollout (one pass) | 2.5–4.0 s per step [ 125 ] , episodes of tens of minutes [ 115 ] , 5–10 min environment re-initialization per training step [ 15 ] | about 20 h per real-world experiment with human resets [ 19 ] , 2.6–16.9 s per judge query [ 21 ] |
| ✜Reported group size | 4 10 [ 7 , 22 , 8 , 126 , 127 ] | 3 16 [ 128 , 129 , 9 ] | 8 64 [ 130 , 115 , 17 , 101 ] | 8 [ 62 ] |
| ② Quality factor : how informative is each rollout? | ||||
| MR | MG | GW | EM | |
| ① Fidelity factor : how faithfully does order trajectories by ? | ||||
| ✜Signal source (provenance) | majority vote a , self-judgment a , computed answers e | self-rewards a , same-model judges a , external judges b | task checkers b , VLM judges b , co-evolving RMs a , same-model judges a | episode outcomes e , progress heads and same-model judges a , VLM judges b |
| ✜Channel coverage | full (discrete answer) | often perceptual only | full on screen, partial for backend state | full in simulation, visual only on hardware |
| ✜Gaming resistance under | high for computed answers, untested for votes | low [ 9 , 129 ] | untested beyond downstream success [ 142 ] | low without judge oversight [ 143 ] |
| ② Localization factor : how finely can credit be apportioned along ? | ||||
| ✜Temporal granularity | episode step ( without tool calls) | episode ( if single-pass) | episode (judges), step (co-evolving RMs) | episode (outcomes), step (progress heads) |
| MR | MG | GW | EM | |
| ① Retention factor : does the cycle- gain survive subsequent updates? | ||||
| ✜Dominant substrate | , except training-free methods | , or , , if training-free | , or , if training-free | , or , , |
| ✜Iteration robustness under | untested beyond a few cycles | compounding hacking within single runs [ 9 , 129 ] | sustained by trajectory replay [ 14 , 17 ] , no forgetting on held-out environments [ 15 ] | fixed-loop continual RL only [ 95 , 161 ] |
| ✜Dominant failure mode | long-cycle drift | compounding reward hacking | distribution shift | forgetting and distribution shift |
| ② Generalization factor : does the gain transfer beyond ? | ||||
| ✜Transfer locus | parameters (any goal) | parameters, or memory and skills (on use) | parameters, or memory and skills (on use) | parameters, or skills and harnesses (on use) |
| Failure mode | Cross-cycle mechanism | Surfaces at | Carried by | Primary mitigation locus |
| ①Compounding reward hacking | An FA fidelity deficit inside ’s calibration support, tolerable per cycle, re-amplifies into a directional bias | UC | Agent edge, into FA if reads | Tighten at FA, or shrink the update step |
| ②Distribution shift | Rollouts drift off ’s calibration support, where its errors are extrapolations | FA | Agent edge, environment path | Recalibrate at FA, or resample in EG |
| ③Forgetting and plasticity loss | A moving goal distribution makes writes to interfere, erasing earlier skills and eroding plasticity [ 192 ] | UC | Environment path, goal edge | Replay earlier goals in EG, or isolate or reset parameters |
| ④Curriculum collapse | contracts the goal support toward mastered regions, causing to decay | EG | Goal edge, or agent edge into | Inject exploration into , or score goals by difficulty |
| ⑤Model collapse | Self-generated training data narrows and erases its tails, driving down under any sampler [ 193 ] | EG | Agent edge | Accumulate real or external data instead of replacing it [ 194 ] |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| ✜ Questions from datasets or fixed generators | |||||||||
| MM-UPT [ 23 ] NeurIPS’25 | b | unlabeled and pre-synthesized questions | – | a | consensus, majority vote | b | GRPO | 1 run | Qwen2.5-VL |
| TTRV [ 171 ] CVPR’26 | b | 20 unlabeled test inputs | – | a | consensus, answer frequency, entropy control | b | GRPO | 1 run | InternVL, Qwen |
| TTRSD [ 210 ] arXiv’26 | b | 20 unlabeled inputs | – | a | consensus, agreement across three views | b | token-selective GRPO | 1 run | InternVL3 and others |
| IQA-SelfEv. [ 211 ] ICLR’26 | b | unlabeled image pairs | – | a | consensus, pairwise majority vote | b | GRPO | 2 | Qwen2.5-VL |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| ✜ Agent-synthesized or retrieved images and videos | |||||||||
| MM-Zero [ 6 ] EMNLP’26 | a | code-rendered images | 4 | a | consensus, majority vote | b | GRPO | 3 | Qwen3-VL, MiMo |
| Active-Zero [ 8 ] arXiv’26 | a | retrieved open-world images | 8 | a | consensus, majority vote | b | GRPO | 3 | Qwen2.5-VL |
| ViCo [ 238 ] NeurIPS’26 | a | edited scene graphs and chart tables | – | e + b | computed or key | b | GRPO and RWR | 5 | Qwen3-VL-8B |
| Tempo-Zero [ 209 ] arXiv’26 | a | program-rendered videos | 8 | e + a | computed, execution | b | GRPO | 5 | Qwen3-VL |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| ✜ Self-verifiers updated with the policy | |||||||||
| SOLACE [ 9 ] CVPR’26 | b | fixed prompt pool | 16 | a | denoising confidence under re-noising probes | b | Flow-GRPO | 1 run | SD3.5-M |
| IRIS [ 129 ] arXiv’25 | b | 553 GenEval prompts | 8 | a | negative self-certainty over text and image tokens | b | GRPO | 1 run | Janus-Pro |
| CoRL [ 13 ] NeurIPS’25 | b | COCO captions paired with images and QA | 8–16 | a + b | own token-embedding alignment, BLIP re-caption and LPIPS cycle consistency | b | joint then task-specific GRPO | 2 runs | Janus-Pro-1B |
| UniRL [ 128 ] arXiv’25 | b | constructed GenEval-category prompts | 3 | a | own answers against answers fixed at prompt construction | b | SFT and GRPO | 1 run | Show-o, Janus |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| RSIgen [ 260 ] arXiv’26 | a | curricula compiled from own probes | – | b | fixed external evaluator | b | agent and generator updates | 3 attempts | BAGEL, Qwen-Image, Wan-Image |
| OctoT2I [ 258 ] CVPR’26 | a | self-proposed prompts over concept dimensions | 5 | b | frozen MLLM probability of alignment | a | routing knowledge base (I) | 1 pass | Qwen2-0.5B router |
| GenEvolve [ 266 ] arXiv’26 | b | prompts with judge-extracted experience | 6 | b | Gemini KScore judge, program-sufficiency judge | a | GRPO, self-distillation from own experience-conditioned targets | – | Qwen3-VL-8B, Qwen-Image-Edit |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| ✜ Stationary sampling of fixed goals | |||||||||
| Mobile-Agent-E [ 162 ] arXiv’25 | b | five-task scenario sequences | – | a | Experience Reflectors over action and error histories s | a | Tips and Shortcuts (M, T) | 1 pass | GPT-4o, frozen |
| Agent S [ 278 ] ICLR’25 | b | exploration tasks, then OSWorld tasks | – | a | self-evaluator summaries of each task | a | narrative and episodic memory (M) | task stream | GPT-4o, frozen |
| MobileSteward [ 279 ] KDD’25 | b | cross-app instructions | – | a | StewardAgent evaluation s | a | expertise and guideline memory (M) | task stream | GPT-4o, frozen |
| AppAgentX [ 280 ] arXiv’25 | b | DroidTask, AndroidWorld, and A3 tasks, five repetitions | – | b | benchmark success evaluation | a | shortcut graph (M, T) | 5 | GPT-4o, frozen |
| Experience Generation (EG) | Feedback Acquisition (FA) | Update Consolidation (UC) | |||||||
| Method | Goal content | Scoring reference & signal | Update | Rounds | Backbone | ||||
| ✜ Environment feedback | |||||||||
| EmbodiedRSI [ 166 ] arXiv’26 | a | experiments chosen by value of information (Sim Real) | – | e | physical trial outcomes | a | code, skills, hierarchical memory (T, M), PPO memory policy | – | frozen VLA, Qwen3.5-122B |
| RoboBridge [ 289 ] arXiv’26 | b | LIBERO-PRO training seeds (Sim Real) | – | e + b | completion predicate, held-out validation gate | a | versioned skill documents (M, I) | 2 | , Codex, frozen |
| EmbodiedEvo [ 290 ] arXiv’26 | b | repeated attempts with scene resets (Sim) | – | e | completion, errors, termination status | a | executable skills, lessons (T, M) | – | coding agent, frozen |
| RoboHarn-Evo [ 291 ] arXiv’26 | b | interaction rollouts on RMBench (Sim, Real) | – | e | physical feedback on rollouts | a | hierarchical physical knowledge (M) | 80 rollouts | GPT-5.5, GPT-6, frozen |
| Mechanism | Trade-off | Degradation | ||||||
| Name | Entry | Raises | Requires | Pays | Path | Mode | Evidence | Instances |
| ✜ Experience generation | ||||||||
| Success-conditioned allocation | fixed task set with checkable outcomes (T) | coverage of tasks dropped by design [ 17 ] | goal | curriculum collapse | untested | MR [ 208 ] , MG [ 11 ] , GW [ 115 , 17 ] , EM [ 21 , 296 ] | ||
| Agent-posed goals | checkable outcomes or output equivalence (T) | pooled keys or scripts, label accuracy 72% to 61% [ 5 ] | agent | curriculum collapse | untested, countered by diversity terms [ 127 , 5 ] | MR [ 5 , 22 ] , MG [ 40 , 260 ] , GW [ 284 , 201 ] , EM [ 302 , 20 ] | ||
| Record-conditioned goal generation | checkable outcomes or output equivalence (T) | pooled keys or scripts | goal | curriculum collapse | untested | GW [ 16 , 15 ] | ||
| Program-checkable goal families | at a program-checked | program, simulator, or scripted scene checks (T) | coverage outside the family | agent or goal | forgetting | 72% to 60% after a vocabulary switch [ 191 ] | MR [ 64 , 235 , 218 ] , MG [ 191 ] , EM [ 304 ] | |
| Benchmark | Goals | Reset | Scorer | Pool | Env | Checker |
| ✜ Multimodal reasoning | ||||||
| MathVista [ 240 ] | 6,141 math questions | none needed | answer key ⋄ | ✓ | – | ✓ |
| MathVerse [ 241 ] | 2,612 math problems, 6 versions | none needed | answer key ⋄ | ✓ | – | – |
| MathVision [ 326 ] | 3,040 math problems, 16 disciplines | none needed | answer key ⋄ | ✓ | – | – |
| MMMU [ 327 ] | 11.5K college questions, 30 subjects | none needed | answer key ⋄ | ✓ | – | ✓ |
| ChartQA [ 242 ] | 32.7K chart questions | none needed | answer key ⋄ | ✓ | – | ✓ |
Multimodal Agents
Multimodal agents perceive and reason across text, images, audio, and video to use tools, interact with interfaces, and act in the world.
See all 409 papersThe Trinity: Capability-Driven Recursive Self-Improvement for Multimodal Agents
AgentStream: How Well Do Self-Evolving LLM Agents Perform Under Streaming Tasks?
Isolated, Sequential, and Interleaved streaming scenarios at test time, which progressively vary the scope and domain composition of the stream. Over these scenarios, we combinatorially evaluate five representative self-evolving methods across three frontier foundation models, disentangling how model capability, method architecture, and streaming scenario jointly shape self-evolution. Our results show that self-evolution reliability varies across streaming scenarios, the benefit of self-evolution is gated by model capability and is non-monotonic in model strength, and no single method dominates across models and scenarios. These findings offer concrete guidance for selecting self-evolving methods across models and streaming scenarios. Overall, we advocate that self-evolving agents should be evaluated under realistic task streams rather than isolated single-task settings.