Code Generation Evaluation

Ask about this topic

Momentum

17 papers in the last four weeks, up 113% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 147

All topics
CardsList

  1. DevIntent: How Much Does LLM-Generated Code Violate Developer Intent?

    Aug 7, 2026Susana Haing, Natan Vidra, Spurthi SettyLanguage Model Generation EvaluationCode Generation Evaluation

    Anote AI

  2. Characterizing the Quality Profile of AI-Generated C++ in Production

    Aug 6, 2026Michael Tran, Fred Lewis, Kun Yang +5Code GenerationCode Generation Evaluation

    Google, Mountain View, USA

  3. Coding Agents as Test-Suite Auditors: Finding What Official Suites Miss While Approaching What They Catch

    Aug 3, 2026Shuyang Xie, Shuxiao Xie, Feng Zhu +2Automated Software TestingBenchmark Auditing

    Harbin Institute of Technology · Fudan University · Sun Yat-sen University

  4. Security-First Evaluation of Text-to-Terraform: Benchmarking LLMs and SLMs for Secure IaC Generation

    Aug 2, 2026Francis Luis Santos Vargas, Rodrigo Brandão Mansilha, Diego KreutzLLM EvaluationCode Generation

    AI Horizon Labs, PPGES, Universidade Federal do Pampa (UNIPAMPA), Alegrete, Brazil

  5. TaPR: Test-Aware Policy Refinement for Feedback-Conditioned Code Generation

    Aug 1, 2026Aofan Liu, Jingxiang Meng, Fangxin Liu +1RL for Code GenerationReward Shaping

    ∗These authors contributed equally. · †Corresponding author.

  6. Simulation Code Generation for Fluid Systems using Large Language Models: Benchmarking Models and Prompting Strategies

    Jul 31, 2026Jan Marius Stürmer, Jascha Knack, Tobias Koch +1LLM PromptingCode Generation Evaluation

    German Aerospace Centre (DLR), Institute for the Protection of Terrestrial Infrastructures, Rathausallee 12, 53757, Sankt Augustin, Germany · Algorithms for Computer Vision, Imaging and Data Analysis, Technische Hochschule Würzburg-Schweinfurt, Ignaz-Schön-Straße 11, Schweinfurt, 97421, Germany

  7. SecDrift: Measuring Sector-Conditioned Security Drift in AI-Generated Code

    Jul 28, 2026Narayanaswami Natraj Bharadwaj, Dhivya ChandramouleeswaranCybersecurityLLM Security

    Independent Researcher USA

  8. RIDGE: An Autonomous Framework for Validation and Method Discovery in LLM-Generated Option Pricing

    Jul 28, 2026Liexin Cheng, Xue Cheng, Shuaiqiang Liu +1Quantitative FinanceCode Generation Evaluation

    Mathematical Institute, Utrecht University · School of Mathematical Sciences, Peking University · Delft Institute of Applied Mathematics, Delft University of Technology

  9. Learning from 53.6K Real-World Developer Edits of AI-Generated Code

    Jul 27, 2026Jenny T. Liang, Mihika Bairathi, Wayne Chi +3Code Generation EvaluationCode Language Models

    Carnegie Mellon University

  10. Adversarial Test-Hardening for AI-Written Code: An Instrument Autopsy and a Pre-Registered Causal Estimate of the Critic Loop

    Jul 25, 2026Jeff OttersonAutomated Software TestingLLM Auditing

    W. P. Carey School of Business, Arizona State University

  11. AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation

    Jul 24, 2026Jie "JW" WuCode GenerationCode Generation Evaluation

    Michigan Technological University, Houghton, MI, USA

  12. The Best Programming Language for Tokenmaxxing: An Investigation of Coding Agent Behavior Across Programming Languages

    Jul 24, 2026Zixuan Wu, Carolyn Jane Anderson, Arjun GuhaAI Coding AgentsCode Generation

    Northeastern University · Wellesley College

  13. MineValiCoder: Reliable Code Generation with Test Case Quality Mining and Bipartite Graph-Based Mutual Validation

    Jul 24, 2026Zhen Zhao, Qihang Yang, Feifei Dai +2Execution-Guided Code GenerationCode Generation

    Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · University of Electronic Science and Technology of China, Chengdu, China

  14. How Do AI Coding Agents Contribute to Software Development? an Empirical Study of Agentic Pull Requests

    Jul 23, 2026Iren Mazloomzadeh, Mohammad Mehdi Morovati, Foutse KhomhSoftware Engineering AgentsAI Coding Agents

    Polytechnique Montréal, Canada

  15. Benchmarking LLMs for Verilog Design Flows

    Jul 23, 2026Angshuman Chakravertty, Rahul Koshti, Buddhi Prakash Sharma +1Code GenerationCode Generation Evaluation

    School of Technology Management and Engineering, NMIMS, Hyderabad, India · Department of Electronics and Communication Engineering, Indian Institute of Technology, Roorkee, India · Department of Electrical and Electronics Engineering and the Anuradha and Prashanth Palakurthi Centre for Artificial Intelligence Research (APPCAIR), Birla Institute of Technology and Science, Pilani (BITS Pilani), Pilani 333031, India

  16. Code Monitor Red Teaming for Public-Test-Passing Code

    Jul 23, 2026Junchi Liao, Jiawen Deng, Fuji RenLLM AuditingCode Generation Evaluation

    University of Electronic Science and Technology of China Chengdu, China

  17. Cross-Model LLM Code Review: Should you use Claude to review Codex or vice versa?

    Jul 22, 2026Zuodong Xiang, Yike Zhang, YueMing Zhang +1Code Generation EvaluationAutomated Code Review

    University of California, Davis USA · Johns Hopkins University USA · California State University, Long Beach USA

  18. SciCodePile: A 128GB Corpus and Executable Benchmark for Challenging Scientific Code Generation

    Jul 21, 2026Weifeng Sun, Ye Fan, Yuchen Chen +6Scientific Code GenerationCode Generation Evaluation

    Singapore Management University · Nanjing University · SUN YAT-SEN University +1

  19. Spaghetti Architect: A Contamination-Resistant, By-Construction-Labelled, Multi-Language Code Dataset Generator

    Jul 21, 2026Yuxiang JiSynthetic Benchmark GenerationData Contamination in Language Models

    Sunway College

  20. The Librarian Who Refused to Code: Model-Dependent Identity Enactment in LLM Code Generation

    Jul 19, 2026Shayell Aharon Salomon, Noam Israel, Ido Safruti +1Personality Modeling in Language ModelsCode Generation

    Bluebear Security

  21. Large Language Models for Code Generation from Multilingual Prompts: A Curated Benchmark and a Study on Code Quality

    Jul 16, 2026Saima Afrin, Alessandro Midolo, Camilo Escobar-Velásquez +5Multilingual Language Model EvaluationCode Generation

    William & Mary, VA, USA · University of Catania, Italy · Universidad de los Andes, Colombia +2

  22. Quantize with Confidence? An Empirical Study of Quantization for Code Generation

    Jul 15, 2026Saima Afrin, Md. Zahidul Haque, Antonio MastropaoloLLM QuantizationCode Generation Evaluation

    AURA Lab · Department of Computer Science · William & Mary +1

  23. Form, Not Content? A Preregistered, Placebo-Controlled Evaluation of Learned Error-Conditioned Self-Repair Through Prompts and Weights in Frozen Small Code Models

    Jul 14, 2026Mehmet IscanLLM Self-CorrectionAutomated Program Repair

    PythaLab, Yıldız Technical University, Istanbul, Turkey

  24. Line-Anchored Feedback Cuts Token Costs and Improves Correctness in AI Code Editing

    Jul 14, 2026William Franz LambertiCode Generation EvaluationCode Language Models

    Computational and Data Sciences College of Science George Mason University Fairfax, VA, United States

  25. Token Reduction Is Not Cost Reduction

    Jul 13, 2026Sarel Weinberger, Amir HozezAI Coding AgentsCost-Aware Inference

    PointFive

  26. When the Reward Suite Is Leaky: A Preregistered Causal Contrast of Natural Verifier False Positives in RLVR

    Jul 13, 2026Chuyifei ZhangAlgorithmic AuditingReinforcement Learning with Verifiable Rewards

    Beijing Jiaotong University

  27. When Does Restricting a Coding Agent to execute_code Help? A Regime ×\times Agent-Design Ablation

    Jul 12, 2026Hong Yang, Qi Yu, Travis DesellCoding AgentsTool-Using Agents

    Rochester Institute of Technology Rochester, New York, USA

  28. Knowledge-Conditioned, Single-Pass LLM Synthesis of Executable Unity Game Scenes: A Compiler Error Census across 26 Goal Playable Concepts

    Jul 11, 2026Hugh Xuechen Liu, Kıvanç TatarLLM-Based Program SynthesisProcedural Content Generation

    Chalmers University of Technology and University of Gothenburg, SE-412 96 Göteborg, Sweden

  29. The Patchwork Problem in LLM-Generated Code

    Jul 9, 2026Viraaji Mothukuri, Reza M. PariziStatic Code AnalysisLLM Reliability

    Decentralized Science Lab, College of Computing and Software Engineering Kennesaw State University, GA, USA

  30. PHITSBench: an execution-scored benchmark for AI-assisted PHITS radiation-transport input generation using natural language

    Jul 8, 2026Xianglin Ji, Svetlana V. BoriskinaKnowledge Augmentation for Language ModelsCode Generation

    Department of Mechanical Engineering, Massachusetts Institute of Technology, Cambridge, MA 02139, USA