LLM Code Generation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 196

All topics

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation
Apr 29, 2026Yeheng Chen, Chaoxiang Xie, Yuling Shi +4Code Generation Synthetic Benchmark Generation
Shanghai Jiao Tong University China
TDD Governance for Multi-Agent Code Generation via Prompt Engineering
Apr 29, 2026Tarlan Hasanli, Shahbaz Siddeeq, Bishwash Khanal +3LLM Prompting AI Agent Governance
Tampere University Tampere, Finland
QASM-Eval: A Dataset to Train and Evaluate LLMs on OpenQASM-3 Beyond Quantum Circuits
Apr 28, 2026Zhenxiao Fu, Lei Jiang, Fan ChenLLM Evaluation Code Generation
Indiana University Bloomington
Defective Task Descriptions in LLM-Based Code Generation: Detection and Analysis
Apr 27, 2026Amal Akli, Mike Papadakis, Maxime Cordy +1Code Generation Evaluation Language Model Robustness
University of Luxembourg Luxembourg
Leveraging LLMs for Multi-File DSL Code Generation: An Industrial Case Study
Apr 27, 2026Sivajeet Chand, Kevin Nguyen, Peter Kuntz +1LLM Fine-Tuning Code Generation
Technical University of Munich · Munich, Germany · BMW Group
How LLMs Fail and Generalize in RTL Coding for Hardware Design?
Apr 26, 2026Guan-Ting Liu, Chao-Han Huck Yang, Chenhui Deng +3LLM Evaluation Code Generation
NVIDIA Research
Tandem: Riding Together with Large and Small Language Models for Efficient Reasoning
Apr 26, 2026Zichuan Fu, Xian Wu, Guojing Li +7Efficient Language Model Reasoning LLM Inference Acceleration
City University of Hong Kong · Tencent Jarvis Lab · Renmin University of China +1
From Natural Language to Verified Code: Toward AI Assisted Problem-to-Code Generation with Dafny-Based Formal Verification
Apr 24, 2026Md Erfan, Md Kamal Hossain Chowdhury, Ahmed Ryan +1LLM Prompting Code Generation
Department of Computer Science, The University of Alabama, Tuscaloosa, USA · Alabama Water Institute, The University of Alabama, Tuscaloosa, USA
BLAST: Benchmarking LLMs with ASP-based Structured Testing
Apr 24, 2026Manuel Alejandro Borroto Santana, Erica Coppolillo, Francesco Calimeri +3Logic Programming LLM Evaluation
University of Calabria, Rende, Italy · ICAR-CNR, Rende, Italy · Artificial Intelligence LAB, DeMaCS, University of Calabria, Rende, Italy +1
You Don't Need Public Tests to Generate Correct Code
Apr 23, 2026Kaushitha Silva, Srinath PereraExecution-Guided Code Generation Code Generation
WSO2 Santa Clara, CA, USA
BODHI: Precise OS Kernel Specification Inference
Apr 22, 2026Zhiming Chang, Ziyang LiKnowledge Augmentation for Language Models LLM Prompting
Department of Applied Mathematics and Statistics Johns Hopkins University Baltimore, US · Department of Computer Science Johns Hopkins University Baltimore, US
WebGen-R1: Incentivizing Large Language Models to Generate Functional and Aesthetic Websites with Reinforcement Learning
Apr 22, 2026Juyong Jiang, Chenglin Cai, Chansung Park +4RL for Code Generation RL for Language Models
The Hong Kong University of Science and Technology (Guangzhou) · Alibaba · Tongyi Lab, Alibaba Group +3
CreativeGame:Toward Mechanic-Aware Creative Game Generation
Apr 21, 2026Hongnan Ma, Han Wang, Shenglin Wang +6Execution-Guided Code Generation Computational Creativity
University of Bristol · Shanghai Jiao Tong University · Shandong University +1
Copy-as-Decode: Grammar-Constrained Parallel Prefill for LLM Editing
Apr 20, 2026Ziyang LiuLLM Inference Acceleration Parallel Decoding
Independent Researcher
Layer-wise MoE Routing Locality under Shared-Prefix Code Generation: Token-Identity Decomposition and Compile-Equivalent Fork Redundancy
Apr 19, 2026Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino +1Expert Routing Efficient Language Model Inference
Graduate School of Informatics, Nagoya University, Japan · Information Technology Center, Nagoya University, Japan
Surgical Repair of Insecure Code Generation in LLMs
Apr 17, 2026Gustavo Sandoval, Brendan Dolan-Gavitt, Siddharth GargMechanistic Interpretability Secure Code Generation
Tandon School of Engineering New York University · XBOW
ACE: Self-Evolving LLM Coding Framework via Adversarial Unit Test Generation and Preference Optimization
Apr 17, 2026Yixu Huang, Xinglei Yu, Zhongyu WeiExecution-Guided Code Generation Language Model Self-Improvement
School of Data Science Fudan University Shanghai, China
HYPERHEURIST: A Simulated Annealing-Based Control Framework for LLM-Driven Code Generation in Optimized Hardware Design
Apr 17, 2026Shiva Ahir, Prajna Bhat, Alex DoboliLarge Language Model-Guided Optimization Code Generation
Department of ECE
QuantCode-Bench: A Benchmark for Evaluating the Ability of Large Language Models to Generate Executable Algorithmic Trading Strategies
Apr 16, 2026Alexey Khoroshilov, Alexey Chernysh, Orkhan Ekhtibarov +2Agentic Code Generation Code Generation Evaluation
Lime
Vibe-Coding: Feedback-Based Automated Verification with no Human Code Inspection, a Feasibility Study
Apr 16, 2026Michal Töpfer, František Plášil, Tomáš Bureš +1LLM Self-Correction Formal Verification
Charles University Prague, Czech Republic
StoryCoder: Narrative Reformulation for Structured Reasoning in LLM Code Generation
Apr 16, 2026Geonhui Jang, Dongyoon Han, YoungJoon YooLLM Prompting LLM Code Generation
Dept. of Artificial Intelligence, Chung-Ang University · NAVER AI Lab
CoDe-R: Refining Decompiler Output with LLMs via Rationale Guidance and Adaptive Inference
Apr 14, 2026Qiang Zhang, Zhongnian LiAdaptive Inference Software Reverse Engineering
School of Computer Science and Technology / School of Artificial Intelligence, China University of Mining and Technology, Xuzhou, China · Mine Digitization Engineering Research Center of the Ministry of Education, China University of Mining and Technology, Xuzhou, China
Bridging Natural Language and Interactive What-If Interfaces via LLM-Generated Declarative Specifications
Apr 8, 2026Sneha Gathani, Sirui Zeng, Diya Patel +5LLM Code Generation Text Generation
University of Maryland, College Park College Park, Maryland, USA · Adobe Research San Jose, California, USA · Microsoft Research Seattle, Washington, USA +2
From Paper to Program: Knowledge Externalization and Bottleneck Diagnosis in AI-Assisted Quantum Many-Body Programming
Apr 5, 2026Yi ZhouScientific Code Generation Code Generation Evaluation
Institute of Physics, Chinese Academy of Sciences, Beijing 100190, China
From Guessing to Placeholding: A Cost-Theoretic Framework for Uncertainty-Aware Code Completion
Apr 2, 2026Liang Zhu, Haolin Chen, Lidong Zhao +1Code Generation LLM Uncertainty Estimation
1Tencent, Shenzhen, China
Embarrassingly Simple Self-Distillation Improves Code Generation
Apr 1, 2026Ruixiang Zhang, Richard He Bai, Huangjie Zheng +3Language Model Decoding Language Model Distillation
Apple
Executing as You Generate: Hiding Execution Latency in LLM Code Interpreters
Apr 1, 2026Zhensu Sun, Zhihao Lin, Zhi Chen +4LLM Inference Acceleration Pipeline Parallelism
Singapore Management University · Beihang University
Design-Specification Tiling for ICL-based CAD Code Generation
Mar 13, 2026Yali Du, San-Zhuo Xi, Hui Sun +1In-Context Learning Text-to-CAD Generation
National Key Laboratory for Novel Software Technology, School of Artificial Intelligence, Nanjing University, China
HardSecBench: Benchmarking the Security Awareness of LLMs for Hardware Code Generation
Jan 20, 2026Qirui Chen, Jingxian Shuai, Shuangwu Chen +8LLM Security Secure Code Generation
University of Science and Technology of China, China · Xi’an Jiaotong University, China · Nanjing University, China +1
X-Coder: Advancing Competitive Programming with Synthetic Tasks, Solutions, and Tests
Jan 11, 2026Jie Wu, Haoling Li, Xin Zhang +8RL for Code Generation Synthetic Data Generation
Tsinghua University · Microsoft

Apr 29, 2026, cs.SEJ/K move · Enter open · S save

ClassEval-Pro: A Cross-Domain Benchmark for Class-Level Code Generation

Yeheng Chen, Chaoxiang Xie, Yuling Shi, Wenhao Zeng+3

Shanghai Jiao Tong University China

Code Generation Synthetic Benchmark Generation Code Generation Benchmarks LLM Code Generation

Open paper AskPDF arXiv

LLMs have achieved strong results on both function-level code synthesis and repository-level code modification, yet a capability that falls between these two extremes -- compositional code creation, i.e., building a complete, internally structured class from a specification -- remains underserved. Current evaluations are either confined to isolated functions or rely on manually curated class-level tasks that are expensive to scale and increasingly susceptible to data contamination. We introduce ClassEval-Pro, a benchmark of 300 class-level tasks spanning 11 domains, constructed through an automated three-stage pipeline that combines complexity enhancement, cross-domain class composition, and integration of real-world GitHub code contributed after January 2025. Every task is validated by an LLM Judge Ensemble and must pass test suites with over 90% line coverage. We evaluate five frontier LLMs under five generation strategies. The best model achieves only 45.6% class-level Pass@1, with a 17.7-point gap between the strongest and weakest models, confirming the benchmark's discriminative power. Strategy choice strongly interacts with model capability: structured approaches such as bottom-up improve weaker models by up to 9.4 percentage points, while compositional generation collapses to as low as 1.3%. Error analysis over 500 manually annotated failures reveals that logic errors (56.2%) and dependency errors (38.0%) dominate, identifying cross-method coordination as the core bottleneck.

Momentum

Related topics

Latest papers 196