cs.SEMar 29, 2026

EffiSkill: Agent Skill Based Automated Code Efficiency Optimization

Authors: Zimu Wang, Yuling Shi, Mengfan Li, Zijun Liu, Jie M. Zhang, Chengcheng Wan, Xiaodong Gu

Organizations: Shanghai Jiao Tong University China · University of California, Berkeley USA · Columbia University USA · King’s College London United Kingdom · 

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Code efficiency is a fundamental aspect of software quality, yet how to harness large language models (LLMs) to optimize programs remains challenging. Prior approaches have sought for one-shot rewriting, retrieved exemplars, or prompt-based search, but they do not explicitly distill reusable optimization knowledge, which limits generalization beyond individual instances. In this paper, we present EffiSkill, a framework for code-efficiency optimization that builds a portable optimization toolbox for LLM-based agents. The key idea is to model recurring slow-to-fast transformations as reusable agent skills that capture both concrete transformation mechanisms and higher-level optimization strategies. EffiSkill adopts a two-stage design: Stage I mines Operator and Meta Skills from large-scale slow/fast program pairs to build a skill library; Stage II applies this library to unseen programs through execution-free diagnosis, skill retrieval, plan composition, and candidate generation, without runtime feedback. Results on EffiBench-X show that EffiSkill achieves higher optimization success rates, improving over the strongest baseline by 3.69 to 12.52 percentage points across model and language settings. These findings suggest that mechanism-level skill reuse provides a useful foundation for execution-free code optimization, and that the resulting skill library can serve as a reusable resource for broader agent workflows.

Figures & tables

Figure 1. Overview of agent skills as reusable knowledge.
Figure 3. Overview of EffiSkill: Skill mining and execution-free optimization.
ModelMethodPythonC++
OPT@1 (%)OPT@8 (%)OPT@1 (%)OPT@8 (%)
GPT-5-miniInstruction18.6231.6231.6248.15
RAG19.9029.3734.1945.43
CoT18.6228.7340.9354.74
SBLLM17.3418.9438.8453.45
FasterPy21.1932.4238.8451.69
Table 1. Optimization performance on EffiBench-X. " ∗ " and " † " denote one-sided paired bootstrap tests against the best non-EffiSkill baseline in the same setting ( p<0.05p<0.05 and p<0.10p<0.10 , respectively).
Figure 4. Optimization success growth from k=1k=1 to k=8k=8 .
Figure 5. Task-level win/loss comparison against the strongest non-EffiSkill baselines.
Figure 6. Fine-grained distribution of top-8 optimization.
ModelMethodOPT@1 (%)OPT@8 (%)
GPT-5-miniEffiSkill26.4837.40
w/o Retrieval12.2027.45
w/ Random Skills13.8027.82
w/o Multi Plans18.7825.84
Qwen3-CoderEffiSkill21.3536.60
w/o Retrieval7.229.15
Table 2. Ablation results on EffiBench-X.
FamilyRepresentative transformationsUsage (%)
Implementation & constant-factorconstant-factor cleanup; lighter data structures; cheaper arithmetic54.2
Algebraic / closed-form reformulationloop elimination; parity / bitwise simplification; range counting19.7
DP / state compressionstate reformulation; rolling-state compression13.1
Combinatorics & number theorymodular combinatorics; coprimality reasoning; binomial simplification9.5
Graph / data structure / set operationsgraph restructuring; set-intersection reformulation; incremental search3.5
Table 3. Skill-family summary across both backbones.
More in this topic

Code Optimization

See all 26 papers

Explore similar work

CardsList
Jul 22, 2026

PerfAgent: Profiler-Guided Iterative Refinement for Repository-Level Code Optimization

Large language model (LLM) agents now perform well on correctness-oriented repository-level tasks, including SWE-Bench issue resolution and feature implementation in real codebases. However, they still struggle with repository-level code optimization, which requires preserving behavior while improving runtime performance. Passing tests is not enough in this setting; a patch must preserve behavior, implement code optimization, and approach expert speedups. Current agents often miss bottlenecks hidden behind abstraction layers and native extensions, stop after shallow speedups, or insufficiently test the code patches that thus may silently break edge cases. We present PerfAgent, a profiler-guided, verifier-in-the-loop workflow that gives an off-the-shelf coding agent the feedback needed to find real hotspots, improve beyond the first passing patch, and use profiler evidence rather than timing alone to decide what to optimize next. On two challenging optimization benchmarks, GSO and SWE-fficiency-Lite, PerfAgent more than doubles the rate of expert-matching patches over OpenHands with GPT-5.1, improving from 19.6% to 39.2% on GSO and from 26% to 74% on SWE-fficiency-Lite. It also surpasses an oracle best-of-five baseline at substantially lower cost, showing that the gains come from better feedback rather than additional test-time sampling.

Ryan Deng, Yuanzhe Liu, Bastian Lipka +4

Massachusetts Institute of Technology, Cambridge, Massachusetts, USA · Rensselaer Polytechnic Institute, Troy, New York, USA · IBM, Ehningen, Germany +2

AI Coding AgentsCode Optimization
AskPDF
Apr 6, 2026

EffiPair: Improving the Efficiency of LLM-generated Code with Differential Execution Feedback

Large language models (LLMs) can generate functionally correct programs that differ substantially in execution efficiency. Existing inference-time optimization methods typically refine each candidate using pointwise runtime or profiling feedback, which identifies how costly an implementation is or where the cost arises, but offers limited guidance on how the computation should change. We introduce DIFFERENTIAL EXECUTION FEEDBACK (DEF), which compares implementations that are nearby in structural program space but separated in performance, turning their execution and implementation differences into directional optimization evidence. We demonstrate DEF in EffiPair, a training-free, test-time framework that pairs structurally similar programs with different efficiencies, distills their relative execution behavior into compact feedback, and iteratively refines a candidate pool. Across EvalPerf, Mercury, and ENAMEL, using GPT-4o mini, DeepSeek-V4.1 Flash, and GPT-5 mini, EffiPair achieves the highest value on each benchmark's official efficiency metric in all nine model-benchmark settings under matched evaluation conditions. Moreover, two contrastive refinement rounds improve efficiency over the selected initial draft in every setting while preserving or improving Pass@1. These results demonstrate the effectiveness of relational execution feedback as a lightweight signal for test-time code optimization.

Samira Hajizadeh, Suman Jana

Department of Electrical Engineering, Columbia University, New York, NY, USA · Department of Computer Science, Columbia University, New York, NY, USA

Inference-Time OptimizationCode OptimizationLLM Code GenerationExecution-Guided Code GenerationDifferential Testing
AskPDF
Jan 31, 2025

LLM Program Optimization via Retrieval Augmented Search

Recent work has demonstrated the potential of large language models (LLMs) for program optimization, a key challenge in programming languages. We propose a blackbox adaptation method called Retrieval Augmented Search (RAS) that performs beam search over candidate optimizations; at each step, it retrieves in-context examples from a given training dataset of slow-fast program pairs to guide the LLM. Critically, we find that performing contextual retrieval based on an LLM-generated natural language description significantly outperforms retrieval based on the source code. We also propose AEGIS, a method for improving interpretability by decomposing training examples into ''atomic edits'' that are significantly more incremental in nature. We show that RAS performs up to 2.06×\times better than prior state-of-the-art blackbox adaptation strategies on optimizing C++ programs, and that AEGIS performs up to 1.37×\times better while making significantly smaller edits. We also show that using RAS improves the mean runtime percentile of Python programs by 10.27 compared to baselines.

Sagnik Anupam, Alexander Shypula, Osbert Bastani

University of Pennsylvania

Large Language Model-Guided OptimizationIn-Context LearningRetrieval-Augmented ICL
AskPDF