Execution-Guided Code Generation

Follow this topic Ask about this topic

Execution-guided code generation uses evidence from running programs—such as test results, compiler errors, runtime traces, and rendered outputs—to help language models produce and repair code. Common approaches iteratively revise a candidate, generate or refine tests, select among candidates by comparing behavior, or train models with execution-based rewards and step-level feedback. These methods are evaluated on settings ranging from programming benchmarks such as HumanEval, MBPP, and LiveCodeBench to repository tasks, database queries, and generated interfaces. A key tension is that execution feedback can catch syntax and runtime failures but may be noisy, costly, or misleading when tests are incomplete or the model and verifier share a mistaken assumption.

Latest papers

67 papers8 papers in the last four weeks, up 100% on the four weeks before. 0.0% of all new papers.

Secure and Functional Code Generation with Self-Evaluation
Oct 24, 2026Md Afif Al Mamun, Gias UddinSecure Code Generation Retrieval-Augmented Code Generation
University of Calgary, Calgary, Canada · York University, Toronto, Canada
Secure Agentic Coding through Counterexample-Grounded Feedback
Oct 19, 2026Tue Le, Anh Nguyen Hoang, Gabriel Sasseville +3Secure Code Generation Agentic Code Generation
Quantum AI & Cyber Security Institute (QACI), FPT Corporation · Mila - Quebec Artificial Intelligence Institute · Université de Montréal +1
3DCodeVerse: Exploring 3D Worlds as Code
Oct 15, 2026Yipeng Gao, Ziyao Zeng, Jinfa Huang +16Agentic Code Generation 3D Asset Generation
University of Southern California · Yale University · University of Rochester +6
HERO: Harnessing Evidence for Test-time Refinement and Prompt Optimization in Program Generation
Oct 13, 2026Mingyu Lu, Timothy Ossowski, Tristan Naumann +2Automatic Prompt Optimization LLM Code Generation
University of Washington · Microsoft Research · Recursion
Verification Trap: Understanding Test-Time Selection Failures under False Premises in Code Generation
Oct 4, 2026Feng He, Hejia Wang, Linghao Meng +2Execution-Guided Code Generation Code Generation Evaluation
University of Science and Technology of China, China · Beijing University of Posts and Telecommunications, China · National University of Singapore, Singapore +1
Why, Where, How: Taxonomy-guided Error Grounding for Code Repair in NL2SQL
Oct 4, 2026Suchan Lee, Woomin Song, Hwanjo Yu +1LLM Self-Correction Execution-Guided Code Generation
POSTECH · KAIST
TeleGen: Improving LLM-Based Web Application Generation via Runtime Telemetry
Oct 4, 2026Yujia Luo, Haonan Zhang, Jiasi Shen +2Web Application Generation Execution-Guided Code Generation
The Hong Kong University of Science and Technology (Guangzhou) · University of Waterloo · The Hong Kong University of Science and Technology
ReMCTS: Reflection-Enhanced Monte Carlo Tree Search for Code Generation
Sep 28, 2026Huifei Wang, Xinying Huang, Yiheng Sun +1Execution-Guided Code Generation LLM-Based Program Synthesis
College of Computer Science and Software Engineering, Shenzhen University · School of Artificial Intelligence, Shenzhen University
SciWalker: Synthesizing Scientific Coding Problems with Operator Graphs and Execution Feedback
Sep 24, 2026Chenxi Li, Wenxuan Zeng, Yun Luo +4Execution-Guided Code Generation Scientific Code Generation
The Hong Kong University of Science and Technology · Shanghai AI Laboratory · Tsinghua University +1
Dynamic Adaptation of the LLM Context for Generating Routines with Coupled Semantics
Sep 3, 2026Gnaneswar Villuri, Hashmath Shaik, Alex DoboliLLM-Based Program Synthesis Execution-Guided Code Generation
Department of ECE, Stony Brook University, Stony Brook, NY, USA
Can Coding Agents Reproduce Official Statistics? Metadata, Retry Budget and the Limits of Execution Feedback in a Controlled Eurostat Benchmark
Sep 2, 2026Sabina-Cristiana NeculaData Analysis Agents AI Agent Benchmarks
Alexandru Ioan Cuza University of Iași, Faculty of Economics and Business Administration, Iași, Romania
Rendering-in-the-Loop: An Execution-Driven Agent for Interactive Web Development
Sep 2, 2026Yilong Guo, Hanqi Chen, Zixiao Ye +3Web Application Generation Execution-Guided Code Generation
Baidu Inc., Beijing, China · School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China
IterCAD: Iterative Program Repair for CAD Code Generation from Orthographic Views
Aug 25, 2026Yuchuan Wu, Ke Niu, Haiyang Yu +3Parametric CAD Modeling Execution-Guided Code Generation
Fudan University Shanghai, China
Iteration Without Elaboration: A Simple ReAct Architecture Suffices for Text-to-SQL Generation
Aug 23, 2026Jian Lu, Haiwei Yu, Raymond M Xiong +2Text-to-SQL Execution-Guided Text-to-SQL
Duke University
Function-Level Execution Feedback for Code Preference Optimization
Aug 23, 2026Idris Nechnech, Sehwan Kim, Jimin Seo +4Code Generation Preference Optimization
Department of Electrical and Computer Engineering, Seoul National University · Department of Computer Science and Engineering, Konkuk University
TDD-Agent: Test-Driven Reasoning for Code Generation
Aug 17, 2026Hongyue Yu, Kefan Li, Jiakun Li +4Code Generation Execution-Guided Code Generation
National College for Excellent Engineers, Beihang University · School of Computer Science and Engineering, Beihang University · Qingdao Research Institute and Hangzhou Innovation Institute, Beihang University +1
VisEditBench: Can Vision-Language Models Edit Visualization Code from Multimodal Feedback?
Aug 11, 2026Mizanur Rahman, Arshia Azimlu, Shadikur Rahman +4VLM Evaluation Data Visualization
‡York University · $Nanyang Technological University · ¶Salesforce AI Research
ExeCRE: Execution-Consistency Guided Reliability Estimation for Self-Correcting Code Generation
Aug 5, 2026Yiru Dong, Richong Zhang, Fanshuang Kong +1LLM Self-Correction Execution-Guided Code Generation
Beihang University Beijing, China
Route-Align-Verify for Functional Correctness in Code Generation
Aug 4, 2026Erxue Zhou, Jingxiang Meng, Aofan LiuExecution-Guided Code Generation Code Generation
Software Engineering Institute, East China Normal University, Shanghai 200062, China · University of Chicago · Peking University
TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation
Aug 1, 2026Aofan Liu, Jingxiang Meng, Fangxin Liu +1RL for Code Generation Reward Shaping
∗These authors contributed equally. · †Corresponding author.
MineValiCoder: Reliable Code Generation with Test Case Quality Mining and Bipartite Graph-Based Mutual Validation
Jul 24, 2026Zhen Zhao, Qihang Yang, Feifei Dai +2Execution-Guided Code Generation Code Generation
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · University of Electronic Science and Technology of China, Chengdu, China
PairCoder++: Pair Programming as a Universal Paradigm for Verified Code-Driven Multimodal and Structured-Artifact Generation
Jul 2, 2026Junhao Chen, Xiang Li, Mingjin Chen +9Execution-Guided Code Generation Code Generation
1THU · 2PKU · 3PolyU, Hong Kong +7
Falsification, Not Exposure: An Internally Preregistered Placebo-Controlled Decomposition of Self-Repair Feedback in Frozen Small Code Models
Jun 30, 2026Mehmet IscanLLM Self-Correction Automated Program Repair
PythaLab, Yıldız Technical University, Istanbul, Turkey
DA-Studio: An Agentic System for End-to-End Data Analysis
Jun 30, 2026Yizhe Liu, Shaolei Zhang, Ju FanExecution-Guided Code Generation Agentic Workflows
Renmin University of China
The Verifier is the Curriculum: Execution-Gated Self-Distillation for Cross-Family Game Generation
Jun 23, 2026Chenyu Zhou, Qiliang Jiang, Shuning Wu +1Execution-Guided Code Generation Language Model Distillation
School of Engineering, Institute of Science Tokyo, Japan · College of Control Science and Engineering, Zhejiang University, China · Department of Electrical and Computer Engineering, National University of Singapore, Singapore
Complexity and Scale in AI-Assisted Workflow Management: A Federated Learning Case Study
Jun 16, 2026Komal Thareja, Hamza Safri, Rajiv Mayani +2Execution-Guided Code Generation Code Generation
RENCI, University of North Carolina at Chapel Hill, NC, USA · Information Sciences Institute, University of Southern California, Marina del Rey, CA, USA
Unlocking LLM Code Correction with Iterative Feedback Loops
Jun 16, 2026Le Zhang, Suresh KothariLLM Self-Correction Execution-Guided Code Generation
Iowa State University, Ames Iowa 50010, USA
AutoMine Solution for AV2 2026 Scenario Mining Challenge
Jun 10, 2026Songliang Cao, Jiele Zhao, Yuru Wang +10VLMs for Autonomous Driving Execution-Guided Code Generation
Xiaomi EV · Huazhong University of Science and Technology
ACE-SQL: Adaptive Co-Optimization via Empirical Credit Assignment for Text-to-SQL
Jun 4, 2026Xiaobing Chen, Ai Jian, Eryu Guo +1Execution-Guided Code Generation Schema Linking
Harbin Engineering University, Harbin, China · Harbin Institute of Technology, Harbin, China · Beijing University of Posts and Telecommunications, Beijing, China
Bootstrapping Semantic Layer from Execution for Text-to-SQL
Jun 4, 2026Youngwon Lee, Jaejin Kim, Seung-won HwangExecution-Guided Code Generation Text-to-SQL
Seoul National University

Oct 24, 2026J/K move · Enter open · S save

Secure and Functional Code Generation with Self-Evaluation

Md Afif Al Mamun, Gias Uddin

University of Calgary, Calgary, Canada · York University, Toronto, Canada

Secure Code Generation Retrieval-Augmented Code Generation Execution-Guided Code Generation Code Generation Evaluation

Open paper AskPDF arXiv

Large Language Models (LLMs) can generate functionally correct code that remains insecure. Security-focused fine-tuning can be expensive and is not always available, motivating inference-time approaches. Security-oriented prompting and retrieval can reduce vulnerabilities, but may also remove required behavior or apply defenses incompatible with the task. The challenge is to identify relevant security knowledge and use it to improve security without sacrificing functionality. We present SecGent, an inference-time framework that retrieves weakness context from the Common Weakness Enumeration (CWE) database and applies it to secure code generation. By connecting task specifications to relevant textual CWE records, SecGent provides task-specific security guidance without relying on retrieved vulnerable or patched code examples. Complementary functional and security tests check the resulting implementation, and their execution feedback guides repairs toward satisfying both objectives. We evaluate SecGent with four open-source LLMs and Claude Sonnet 5.5 across five programming languages on the CWEval benchmark. It achieves the highest joint functional--security success against four inference-time baselines for all five models. Across the open-weight models, mean functional-security@1 (FS@1) improves by 15.5% over RAG, with gains in both functionality and security, and by 19.8% over the strongest prompt optimizer compiled for each model. On Sonnet, FS@1 reaches 92.1%, outperforming the strongest baseline AutoGen by 4.8%. In a separate comparison, SecGent outperforms Claude Code with security-focused prompting by over 6% in FS@1. Feedback-guided repair further improves FS@1 by up to 6.7% over generation without feedback, primarily by recovering functionality while checking security.