LLM Code Generation

LLM: Large Language Model

This field studies how large language models generate, edit, translate, and test programs from natural-language instructions, examples, or existing code. Common approaches include prompting and retrieval, fine-tuning and reinforcement learning, constrained decoding, and agentic loops that use execution, tests, or static analysis to refine outputs. Evaluations range from HumanEval, MBPP, and LiveCodeBench to repository-level and domain-specific tasks, increasingly measuring security, efficiency, and adherence to requirements alongside whether code runs. Models have improved on standard benchmarks, but reliability on unfamiliar APIs, complex projects, specialized languages, and implicit requirements remains a challenge, as do trade-offs among correctness, safety, cost, and maintainability.

Latest papers

314 papers23 papers in the last four weeks, down 12% on the four weeks before. 0.1% of all new papers.

SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models
Sep 17, 2025Kerui Huang, Shuhan Liu, Xing Hu +3Efficient Language Model Reasoning CoT Compression
The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China · State Key Laboratory for Novel Software Technology, Nanjing University, China
Enhancing Knowledge Density via Hierarchical Context Pruning for Retrieval-Augmented Repository-Level Code Completion
Aug 13, 2025Xiaohan Chen, Xiang Chen, Yu Tian +3LLM Code Generation Retrieval-Augmented Generation
MIIT Key Laboratory of Pattern Analysis and Machine Intelligence, College of Computer Science and Technology, Nanjing University of Aeronautics and Astronautics · Tsinghua University
Experimental Analysis of Productive Interaction Strategy with ChatGPT: User Study on Function and Project-level Code Generation Tasks
Aug 6, 2025Sangwon Hyun, Hyunjun Kim, Jinhyuk Jang +2LLM Prompting Code Generation
Adelaide University, Adelaide, SA, Australia · Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea
RedCoder: Automated Multi-Turn Red Teaming for Code LLMs
Jun 25, 2025Wenjie Jacky Mo, Qin Liu, Xiaofei Wen +5Adversarial Prompt Generation LLM Red Teaming
University of California, Davis · University of Southern California
ChatModel: Automating Reference Model Design and Verification with LLMs
Jun 18, 2025Jianmin Ye, Tianyang Liu, Qi Tian +3LLM-Based Program Synthesis LLM Code Generation
National Center of Technology Innovation for EDA, China · School of Integrated Circuits, Southeast University, China
EffiBench-X: A Multi-Language Benchmark for Measuring Efficiency of LLM-Generated Code
May 19, 2025Yuhao Qing, Boyu Zhu, Mingzhe Du +9Code Generation Benchmarks LLM Code Generation
HKU · UCL · NTU +6
BigO(Bench): Can LLMs Generate Code with Controlled Time and Space Complexity?
Mar 19, 2025Pierre Chambon, Baptiste Roziere, Benoit Sagot +1Benchmark Design Code Generation
FAIR at Meta · Inria · Work done at Meta, now working at Mistral AI
Ensemble Learning for Large Language Models in Text and Code Generation: A Survey
Mar 13, 2025Mari Ashiga, Wei Jie, Fan Wu +5Mixture-of-Experts Language Models Language Model Ensembles
School of Computing and Engineering, University of West London, London W5 5RF, United Kingdom · Turing Intelligence Technology Limited, London EC2M 2PF, United Kingdom · School of Computer Science, University of Leeds, Leeds LS2 9JT, United Kingdom
Evaluating Harmful Memorization in LLM Code Generation
Mar 4, 2025Lizhe Zhang, Wentao Chen, Li Zhong +3Memorization in Language Models LLM Code Generation
University of California, San Diego†
Evaluating LLM-Based Regression Test Generation
Jan 19, 2025Jing Liu, Seongmin Lee, Eleonora Losiouk +1Automated Test Generation LLM Code Generation
MPI-SP, Germany · University of California at Los Angeles, USA · University of Padua, Italy
Leveraging Metamemory Agent for Enhanced Data-Free Code Generation in Large Language Models
Jan 14, 2025Shengsheng Zhou, Shuai Wang, Liang Ding +5Code Generation Agentic Code Generation
College of Computing and Data Science, Nanyang Technological University, Singapore · School of Computer Science, National Engineering Research Center for Multimedia Software, Wuhan University, Wuhan, China · The University of Sydney, Sydney, Australia +3
Can Language Models Replace Programmers for Coding? REPOCOD Says 'Not Yet'
Oct 29, 2024Shanchao Liang, Yiran Hu, Nan Jiang +1LLM Code Generation Code Generation Evaluation
Purdue University
Grammar-Aligned Decoding
May 31, 2024Kanghee Park, Jiayu Wang, Taylor Berg-Kirkpatrick +2Constrained Generative Modeling Constrained Decoding
University of Wisconsin-Madison · University of California San Diego
Leveraging High-Level Synthesis and Large Language Models to Generate, Simulate, and Deploy a Uniform Random Number Generator Hardware Design
Nov 6, 2023James T. MeechElectronic Design Automation RTL Code Generation
University of Cambridge

Sep 17, 2025J/K move · Enter open · S save

SEER: Self-Enhancing Chain-of-Thought Compression for Reasoning Models

Kerui Huang, Shuhan Liu, Xing Hu, Tongtong Xu+2

The State Key Laboratory of Blockchain and Data Security, Zhejiang University, China · State Key Laboratory for Novel Software Technology, Nanjing University, China

Efficient Language Model Reasoning CoT Compression Large Reasoning Models Overthinking in Language Models LLM Code Generation

Open paper AskPDF arXiv

Chain-of-Thought (CoT) prompting can substantially improve the reasoning ability of large language models (LLMs), but it often comes with high inference cost due to long and poorly controlled reasoning traces. This overhead is particularly problematic in software engineering tasks (e.g., code generation), where both latency and output reliability matter. To better understand this trade-off, we conduct an empirical study on widely used code generation benchmarks and observe that many modern reasoning models produce excessively verbose CoTs (often thousands of tokens), which frequently leads to truncation and unstable generation. Using a strict n-gram repetition detector, we find that most observed truncations are associated with degenerate looping behaviors. In addition, a HumanEval/129 case study shows that failed generations can be longer than successful ones, suggesting limited returns from overlong reasoning. Motivated by these findings, we propose SEER (Self-Enhancing Efficient Reasoning), a self-enhancing framework for adaptive CoT compression. SEER improves the conciseness of reasoning while preserving output quality, without relying on external compression tools. SEER refines self-generated CoT data via Best-of-N sampling to suppress looping and redundant traces, then applies a lightweight, data-driven filter to encourage concise yet correct reasoning. It then fine-tunes the model on the filtered data to internalize concise reasoning behaviors. Across four software engineering benchmarks on the evaluated DeepSeek-R1-Distill-Qwen-7B backbone, SEER reduces CoT length by 34.6% on average while improving task performance, with reduced truncation and fewer reasoning loops.