Code Generation Benchmarks

Ask about this topic

Momentum

15 papers in the last four weeks, up 275% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 104

All topics
CardsList

  1. TaoD2C-Bench: Benchmarking MLLMs for Industrial UI Code Generation Beyond Visual Fidelity

    Oct 7, 2026Chengwei Shi, Yunnong Chen, Tingting Zhou +5Code GenerationCode Generation Evaluation

    Zhejiang University · Taobao & Tmall Group of Alibaba

  2. From Pixel to Coding: Evaluating the Figure Reproduction Capabilities of MLLMs

    Oct 7, 2026Zijian Chen, Zhengyu Chen, Bohan Liang +7Multimodal Large Language ModelsMultimodal Model Evaluation

    Institute of Image Communication and Information Processing, Shanghai Jiao Tong University, Shanghai, 200240, China. · Shanghai Artificial Intelligence Laboratory, Shanghai, 200030, China. · Macao Polytechnic University, Macao, 999078, China.

  3. Large-scale Repository Engineering via Agent-Native Reusable Code Primitives

    Oct 6, 2026Haibo Jin, Peng Kuang, Xucheng Yu +3Repository-Level Code GenerationSoftware Engineering

    University of Illinois Urbana-Champaign, USA

  4. GNN-CB: A Graph Neural Network Competition Benchmark for Human and LLM Evaluation

    Oct 4, 2026Murad Hossen, Tasneem Selim, Gurur Gamgam +22LLM EvaluationGraph Neural Networks

    University of Houston, Texas,USA · BASIRA Lab, Department of Computing, Imperial College London, United Kingdom · Department of Mathematics and Computer Science, Faculty of Science, Alexandria University, Alexandria, Egypt +23

  5. VHDL-REPOBENCH: A Repository-Level Benchmark for Evaluating Large Language Models on VHDL Design Generation

    Oct 4, 2026Prashanth Vijayaraghavan, Akul Malhotra, Ashutosh Jadhav +2LLM EvaluationCode Generation

    IBM Research San Jose, CA, USA

  6. Self-Spec Verifiable Code Generation

    Sep 30, 2026Jiaru Qian, Yihong Dong, Yongmin Li +3LLM Self-CorrectionCode Generation

    School of Computer Science, Peking University · Beijing Key Laboratory of Trustworthy Code Large Language Models · Key Laboratory of High Confidence Software Technologies, Peking University, Ministry of Education +3

  7. E2E-SWE: Benchmarking LLMs on Building Working Codebases from Scratch

    Sep 29, 2026Hantian Ding, Chloe Bi, Jiacheng Zhu +5Software Engineering AgentsRepository-Level Code Generation

    Meta Superintelligence Labs

  8. Zero2Repo: Can Coding Agents Build Repositories from Scratch?

    Sep 29, 2026Pei Yang, Tianyu Shi, Yuhang Yao +23Coding AgentsAI Agent Benchmarks

    Gradient Data · McGill University · Carnegie Mellon University +6

  9. LoLBench: Evaluating Coding Agents with Long-Horizon Proposals on Large Software Systems

    Sep 29, 2026Yun Peng, Zihan Wu, Zeyang Zhuang +6Software EngineeringAI Coding Agents

    Fudan University, China · City University of Hong Kong, Hong Kong · Chinese University of Hong Kong, Hong Kong +5

  10. GPUPhysBench: Benchmarking Coding Agents for Correct and Efficient GPU Physics Simulation

    Sep 28, 2026Yuchen Sun, Jinjin He, Sinan Wang +1GPU Kernel OptimizationBenchmark Design

    Georgia Institute of Technology

  11. Do Coding Agents Reuse Existing Code or Reinvent the Wheel?

    Sep 28, 2026Dongsheng Ma, Sizhe Wang, Xinyi Huang +5Coding AgentsCode Generation

    Peking University · Fudan University · Shanghai Jiao Tong University +3

  12. MetaBench-Harness: Unlocking End-to-End Optimization of Benchmark Harnesses

    Sep 27, 2026Xuanjun Chen, Hua-Hsuan Chen, Wei-Chung Lu +3LLM EvaluationBenchmark Design

    National Taiwan University · NTU AI-Core · Queen Mary University of London

  13. GameLogicBench: Evaluating Coding Agents on Runtime Game Logic with Tick-Level State Assertions

    Sep 18, 2026Xinyu Che, Yunfei Ge, Shihao Li +9AI Coding AgentsBenchmark Design

    Nanjing University

  14. SWE-Proof: Can Language Models Resolve Real-World Issues with Machine-Checked Proofs?

    Sep 18, 2026George Ma, Benjamin Mikek, Haoyu Li +9Formal VerificationAgentic Code Generation

    UC Berkeley · Georgia Tech · UIUC +1

  15. What is the Difference Between Me and You? Benchmarking the Quality Gap Between Human-Written and AI-Generated Code

    Sep 14, 2026Cristina Improta, Pietro Liguori, Domenico CotroneoLLM SecurityCode Generation Evaluation

    University of Naples Federico II, Naples, Italy · University of North Carolina at Charlotte, Charlotte, NC, USA

  16. IdeaAMBIG: Benchmarking Implementation-Critical Gaps in Research-Idea Specifications

    Sep 9, 2026Yiling Ma, Yilun Zhao, Sihong Wu +2Benchmark DesignClarification Question Generation

    Yale University · TCS Research

  17. Retrofitting Code Using LLMs to Support Exceptional Behavior

    Sep 9, 2026Linghan Zhong, Jiyang Zhang, Jayanth Srinivasa +2Code GenerationCode Generation Benchmarks

    The University of Texas at Austin, USA · Cisco Systems, USA

  18. SciFigure2Code: An AI-Reconstructed Benchmark for Scientific Figure-to-Code

    Sep 8, 2026Wentao Li, Yibo Wu, Yizhe Chen +5Scientific VisualizationCode Generation Evaluation

    Tsinghua University

  19. GVS5H: Zero-Shot Self-Orchestration with Ledger-Based Control for Improved LLM Coding Performance

    Aug 27, 2026Victor Gao, Vida Khosrowshahi, Ali Khosrowshahi +4Zero-Shot LearningMulti-Agent LLM Systems

    Sang Won · Persis Capital Inc.

  20. Vero: Can AI Agents Build Formally Verified Software Repositories?

    Aug 13, 2026Zhe Ye, Hantao Lou, Yuechun Sun +8Code GenerationFormal Verification

    1UC Berkeley · University of Chicago · 3California Institute of Technology +3

  21. P3^{3}: Joint Program-and-Proof Planning for Verified Code Generation

    Aug 10, 2026Zenan Li, Ziran Yang, Peiyang Song +2Code GenerationFormal Verification

    1Apodex · 2Princeton University · 3Caltech +1

  22. A Unified Issue Resolution Benchmark for Requirement Clarification, Planning, and Code Generation for Coding Agents

    Aug 10, 2026Xin Zhou, Chun Yong Chong, Kisub Kim +11Requirements EngineeringAI Coding Agents

    Singapore Management University, Singapore · Monash University Malaysia, Malaysia · Daegu Gyeongbuk Institute of Science and Technology (DGIST), Republic of Korea +5

  23. DevIntent: How Much Does LLM-Generated Code Violate Developer Intent?

    Aug 7, 2026Susana Haing, Natan Vidra, Spurthi SettyLanguage Model Generation EvaluationCode Generation Evaluation

    Anote AI

  24. SciCode-Verified: How Benchmark Defects Underestimated the Scientific-Coding Ability of Language Models

    Aug 5, 2026Sihan Hu, Lyuhan Huang, Youjin Deng +1LLM EvaluationScientific Code Generation

    Hefei National Laboratory College of Mechanical and Electrical Engineering University of Science and Technology of China · Hefei National Laboratory Institute of Theoretical Physics University of Science and Technology of China Chinese Academy of Sciences Hefei 230026, China · Chinese Academy of Sciences Beijing 100190, China

  25. MT-Web2Code: Benchmarking Coding Agents on Multi-Turn Regional Reconstruction and Localized Modification

    Aug 4, 2026Qiming Li, Shujie Hu, Haohan Liu +3Code GenerationAI Agent Benchmarks

    1Harbin Institute of Technology · 2Meituan, Longcat Team · 3Peng Cheng Laboratory

  26. FinHardBench: Can LLMs Generate Latency-Aware Hardware for Financial Computing?

    Aug 2, 2026Weimin Fu, Hejia Zhang, Minghao Shao +6Large Language Model-Guided OptimizationElectronic Design Automation

    Lehigh University · University of California, San Diego · NYU Tandon School of Engineering +1

  27. EduPluginBench: Executable Assurance for AI-Generated Educational Plugins

    Aug 1, 2026Nizam KadirSoftware SecurityRuntime Assurance

    Singapore University of Technology and Design, Science, Mathematics and Technology, Singapore, Singapore

  28. Change2Task: From Repository Changes to Executable Coding Agent Tasks and Environments

    Jul 30, 2026Haomin Qi, Xingliang Wang, Xuanqi Gao +9Code Generation BenchmarksAgent Benchmarks

    1Microsoft · 3Zhejiang University · 4Xi’an Jiaotong University +1

  29. Fewer Clarifications, Better Code: Benchmarking Cross-Session Personalized Ambiguity Adaptation in Coding Assistants

    Jul 29, 2026Zijian Xu, Wenshuo Zhang, Zisen Qin +4Knowledge Augmentation for Language ModelsCode Generation

    Southeast University · The Hong Kong University of Science and Technology

  30. AssumptionMiner: Extracting, Tracing, and Revising Implicit Assumptions in LLM Code Generation

    Jul 24, 2026Jie "JW" WuCode GenerationCode Generation Evaluation

    Michigan Technological University, Houghton, MI, USA