While "Intent-oriented programming" (or "Vibe Coding") redefines software engineering, existing code agents remain tethered to static code snapshots. Consequently, they struggle to model the critical information embedded in the temporal evolution of projects, failing to leverage the "reasoning trajectories" implicit in past successful practices. This limitation results in rigid behavioral logic and a lack of autonomous adaptability, ultimately hindering their ability to tackle complex, repository-level problems. To bridge this static-dynamic mismatch, we propose MemCoder, a framework designed to enable continual human-AI co-evolution. MemCoder first structures historical human experience to distill latent intent-to-code mappings from past commits. It then employs a self-refinement mechanism driven by verification feedback to correct agent behavior in real-time. Crucially, an experience self-internalization mechanism is introduced to crystallize human-validated solutions into long-term knowledge, thereby supporting sustained evolution. Experimental results on SWE-bench Verified demonstrate that MemCoder not only achieves State-of-the-Art (SOTA) performance but also delivers a 9.4% improvement in resolved rate over the general foundation model DeepSeek-V3.2. These findings indicate that equipping agents with the capability to co-evolve with humans via project history and real-time feedback effectively unlocks the potential of general models in complex software engineering tasks.
Figures & tables
Figure 1: Comparison of MemCoder with existing methods. MemCoder facilitates evolution by learning the intrinsic mapping from high-level intent to concrete code implementation, derived from structured memory.
Figure 2: Architectural overview of MemCoder, illustrating a closed-loop human–AI co-evolution paradigm.In Stage 1, MemCoder reconstructs developer cognition by distilling raw commit histories into structured long-term memory, capturing latent intent-to-code mappings from historical human practices. In Stage 2, the agent performs context-aware dual-stage retrieval to access relevant experience, while a Refining Sub-agent enables execution-time self-refinement through prompt concretization, automated test generation, and verification feedback. Crucially, human-validated solutions are subsequently internalized into long-term memory, closing the evolutionary loop and enabling the agent to progressively align with repository-specific conventions across iterations.
Figure 3: Comparison of MemCoder with the top 6 methods on the SWE-bench Verified leaderboard as of January 20, 2026.
Method
Setting
Resolved(%)
MemCoder + GPT-5.2
pass@ 2
83.8 (419)
MemCoder + GPT-5.2
pass@ 1
78.8 (394)
MemCoder + DeepSeek-V3.2
pass@ 1
77.8 (389)
OpenHands + Claude Opus 4.5
pass@ 3
77.6 (388)
OpenHands + Claude Sonnet 4.5
pass@ 3
74.6 (373)
OpenHands + GPT-5.2
pass@ 3
74.4 (372)
Table 1: Comparative analysis of LLM performance under OpenHands vs. MemCoder on SWE-bench Verified.
Method
Resolved(%)
Δ
MemCoder
77.8 (389)
-
w/o DSR
76.4 (382)
-1.4%(-7)
w/o DSR & ER
73.0 (365)
-4.8%(-24)
w/o CR
71.6 (358)
-6.2%(-31)
w/o all
68.4 (342)
-9.4%(-47)
Table 2: Ablation study of MemCoder on the SWE-bench Verified dataset using DeepSeek-V3.2. The notation ‘w/o’ indicates experiments where specific modules were removed. The abbreviations CR, ER, and DSR denote the commit retrieval module, experience representation module, and the dynamic self-refine module, respectively.
Figure 4: Performance of MemCoder across various top-k values. The metrics include resolved rate, average number of retrieved historical experiences, and average tool call frequency. All experiments are conducted using DeepSeek-V3.2 as the backbone model and evaluated on a randomly sampled 200-instance subset of the SWE-bench Verified dataset.
Figure 5: Performance of MemCoder across various top-k values with the retrieval frequency restricted to one. The metric displayed is the Resolve Rate (%). All experiments are conducted using DeepSeek-V3.2 as the backbone model and evaluated on a randomly sampled 200-instance subset of the SWE-bench Verified dataset.
LLM-driven software engineering agents have become a central testbed for real-world language-model capability, yet their training remains limited by the availability of high-quality SWE tasks. Existing synthetic data methods typically create tasks through fixed mutation or bug-injection procedures, making the resulting distributions largely independent of the agent's own weaknesses and training progress. We introduce Socratic-SWE, a closed-loop self-evolution framework that reuses the agent's historical solving traces as a source of training signal. Rather than treating traces only as evidence for reward computation, Socratic-SWE distills them into structured agent skills that summarize recurring failures and effective repair patterns. These skills then guide the generation of targeted repair tasks in real repositories. Candidate tasks are checked through execution-based validation and scored with a solver-gradient alignment reward, so that the retained tasks are both verifiable and useful for improving the Solver. The updated Solver produces new traces, enabling the task curriculum to adapt over successive rounds. Across SWE-bench Verified, SWE-bench Lite, SWE-bench Pro, and Terminal-Bench 2.0, Socratic-SWE consistently improves over self-evolving baselines under the same compute budget, reaching 50.40% on SWE-bench Verified after three iterations. These results suggest that solving traces can serve as a scalable substrate for self-evolving SWE agents.
Chuan Xiao, Zhengbo Jiao, Shaobo Wang +5
2Shanghai Jiao Tong University · 1AI Data, Alibaba Group
Large Language Models (LLMs) have demonstrated significant potential as autonomous software engineering (SWE) agents. Recent work has further explored augmenting these agents with memory mechanisms to support long-horizon reasoning. However, these approaches typically operate at a coarse instance granularity, treating the entire problem-solving episode as the atomic unit of storage and retrieval. We empirically demonstrate that instance-level memory suffers from a fundamental granularity mismatch, resulting in misguided retrieval when tasks with similar surface descriptions require distinct reasoning logic at specific stages. To address this, we propose Structurally Aligned Subtask-Level Memory, a method that aligns memory storage, retrieval, and updating with the agent's functional decomposition. Extensive experiments on SWE-bench Verified demonstrate that our method consistently outperforms both vanilla agents and strong instance-level memory baselines across diverse backbones, improving mean Pass@1 over the vanilla agent by +4.7 pp on average (e.g., +6.8 pp on Gemini 2.5 Pro). Performance gains grow with more interaction steps, showing that leveraging past experience benefits long-horizon reasoning in complex software engineering tasks.
Large language model (LLM)-powered agents have demonstrated strong capabilities in automating software engineering tasks such as static bug fixing. However, in the real world, the development of mature software is typically predicated on complex requirement changes and long-term feature iterations -- a process that static, one-shot repair paradigms fail to capture. To bridge this gap, we propose SWE-CI, the first repository-level benchmark built upon the Continuous Integration loop, aiming to shift the evaluation paradigm for code generation from static, short-term functional correctness toward dynamic, long-term maintainability. The key insight is simple: Maintainability can be revealed by tracking how functional correctness changes over time. The benchmark comprises 100 tasks, each deriving from a real-world code repository with a development history spanning an average of 233 days and 71 consecutive commits. SWE-CI requires agents to systematically resolve these tasks through dozens of rounds of analysis and coding iterations. SWE-CI provides valuable insights into how well agents can sustain code quality throughout long-term evolution.
Jialong Chen, Xander Xu, Hu Wei +2
Sun Yat-sen University · Skylenage · Alibaba Group