LLM Code Generation

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

17 papers in the last four weeks, up 42% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 196

All topics

PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning
May 19, 2026Qiran Zhang, Yuheng Wang, Runde Yang +9Spatial Reasoning Code Generation Benchmarks
School of Artificial Intelligence, Shanghai Jiao Tong University
AI4BayesCode: From Natural Language Descriptions to Validated Modular Stateful Bayesian Samplers
May 18, 2026Jungang Zou, Alex Ziyu Jiang, Qixuan ChenMarkov Chain Monte Carlo Bayesian Inference
Department of Biostatistics, Columbia University
AutoVecCoder: Teaching LLMs to Generate Explicitly Vectorized Code
May 18, 2026Shangzhan Li, Xinyu Yin, Xuanyu Jin +8Large Language Model-Guided Optimization Code Generation
Harbin Institute of Technology, Harbin, China · Xiamen University, Xiamen, China · Tsinghua University, Beijing, China
The Range Shrinks, the Threat Remains: Re-evaluating LLM Package Hallucinations on the 2026 Frontier-Model Cohort
May 16, 2026Aleksandr ChurilovLLM Evaluation Hallucination in Language Models
Independent Researcher
Task Abstention for Large Language Models in Code Generation
May 16, 2026Yanke Zhou, Yuhao Tan, Senrong Xu +4Selective Prediction LLM Hallucination Mitigation
State Key Laboratory of Novel Software Technology Nanjing University · School of Computing and Mathematical Sciences University of London
Syntax Without Semantics: Teaching Large Language Models to Code in an Unseen Language
May 15, 2026Vinayshekhar Bannihatti Kumar, Disha Makhija, Manoj Ghuhan Arivazhagan +1LLM Fine-Tuning Code Language Models
AWS AI Labs
FrontierSmith: Synthesizing Open-Ended Coding Problems at Scale
May 14, 2026Runyuan He, Qiuyang Mang, Shang Zhou +14Synthetic Benchmark Generation Synthetic Data Generation for Language Models
UC Berkeley · UC San Diego · University of Washington +4
Hydra: Efficient, Correct Code Generation via Checkpoint-and-Rollback Support
May 14, 2026Alexander Du, Jianjun Ou, Danyang Zhuo +1Automated Program Repair Code Generation
Duke University Durham, NC, USA
The Readability Spectrum: Patterns, Issues, and Prompt Effects in LLM-Generated Code
May 13, 2026Hengzhi Ye, Fengyuan Ran, Weiwei Xu +1LLM Prompting Code Generation Evaluation
Peking university · Beijing, Beijing, China · Wuhan University +1
PerfCodeBench: Benchmarking LLMs for System-Level High-Performance Code Optimization
May 13, 2026Huihao Jing, Wenbin Hu, Shaojin Chen +5Benchmark Design High-Performance Computing
HKUST · NYU · SWUPL
Uncertainty Quantification for LLM-based Code Generation
May 12, 2026Senrong Xu, Yuhao Tan, Yanke Zhou +6Uncertainty Quantification LLM Uncertainty Estimation
State Key Lab of Novel Software Technology, Nanjing University · ETH Zürich · Birkbeck, University of London
LLMs for Secure Hardware Design and Related Problems: Opportunities and Challenges
May 11, 2026Johann Knechtel, Ozgur Sinanoglu, Ramesh KarriElectronic Design Automation Cybersecurity
New York University Abu Dhabi · NYU Tandon School of Engineering
CodeClinic: Evaluating Automation of Coding Skills for Clinical Reasoning Agents
May 10, 2026Timothy Ossowski, Xinchi Liu, Danyal Maqbool +6Healthcare LLM Agent Skill Learning
University of Wisconsin–Madison · 2Microsoft Research
SmartEval: A Benchmark for Evaluating LLM-Generated Smart Contracts from Natural Language Specifications
May 10, 2026Abhinav Goel, Agostino Capponi, Alfio Gliozzo +1Code Generation Code Generation Evaluation
Columbia University New York, NY, USA · IBM T.J. Watson Research Center Yorktown Heights, NY, USA
PrepBench: How Far Are We from Natural-Language-Driven Data Preparation?
May 9, 2026Jingzhe Xu, Rui Wang, Jiannan Wang +1LLM Evaluation LLM Code Generation
Department of Computer Science and Technology, BNRist, Tsinghua University
Semantic Voting: Execution-Grounded Consensus for LLM Code Generation
May 9, 2026Shan Jiang, Zijian Yi, Chenguang ZhuExecution-Guided Code Generation Code Generation Evaluation
The University of Texas at Austin
SecureForge: Finding and Preventing Vulnerabilities in LLM-Generated Code via Prompt Optimization
May 8, 2026Houjun Liu, Lisa Einstein, John Yang +5LLM Auditing Prompt Optimization
Stanford University
Mage: Multi-Axis Evaluation of LLM-Generated Executable Game Scenes Beyond Compile-Pass Rate
May 8, 2026Hugh Xuechen Liu, Kıvanç TatarCode Generation Code Generation Evaluation
Chalmers University of Technology and University of Gothenburg
PaT: Planning-after-Trial for Efficient Test-Time Code Generation
May 8, 2026Youngsik Yoon, Sungjae Lee, Seockbean Song +3Test-Time Optimization Execution-Guided Code Generation
Department of Computer Science and Engineering, POSTECH, South Korea · Graduate School of Artificial Intelligence, POSTECH, South Korea · Microsoft Research Asia, Beijing, China
Correct Code, Vulnerable Dependencies: A Large Scale Measurement Study of LLM-Specified Library Versions
May 7, 2026Chengjie Wang, Jingzheng Wu, Xiang Ling +2Software Engineering LLM Security
Intelligent Software Research Center, Institute of Software, Chinese Academy of Sciences, Beijing, China · Key Laboratory of System Software (Chinese Academy of Sciences), Beijing, China
KernelBenchX: A Comprehensive Benchmark for Evaluating LLM-Generated GPU Kernels
May 6, 2026Han Wang, Jintao Zhang, Kai Jiang +3GPU Kernel Optimization GPU Acceleration
∗Equal contribution.
Delta-Based Neural Architecture Search: LLM Fine-Tuning via Code Diffs
May 6, 2026Santosh Premi Adhikari, Radu Timofte, Dmitry IgnatovLarge Language Model-Guided NAS LLM Fine-Tuning
Computer Vision Lab, CAIDAS & IFI, University of Würzburg, Germany
Bridging Generation and Training: A Systematic Review of Quality Issues in LLMs for Code
May 6, 2026Kaifeng He, Xiaojun Zhang, Peiliang Cai +7Training Data Curation Code Generation Evaluation
Sun Yat-sen University, China · Nanyang Technological University, Singapore, Singapore · Tongji University, Shanghai, China +1
Measuring and Mitigating Bias in Code Generated by Large Language Models
May 5, 2026Yuxi Chen, Yutian Tang, Timothy StorerAlgorithmic Bias Language Model Bias Evaluation
School of Computing Science, University of Glasgow, United Kingdom.
Evaluating Prompting and Execution-Based Methods for Deterministic Computation in LLMs
May 4, 2026Hongkun YuLLM Evaluation LLM Prompting
Virginia Tech Blacksburg, VA, USA
AI-Generated Smells: An Analysis of Code and Architecture in LLM and Agent-Driven Development
May 4, 2026Yuecai Zhu, Nikolaos Tsantalis, Peter C. RigbySoftware Engineering LLM Auditing
Department of Computer Science and Software Engineering Concordia University Montréal, Québec, Canada
LLM-Assisted Repository-Level Generation with Structured Spec-Driven Engineering
May 4, 2026Shuzhao Feng, Boqi Chen, Brett H Meyer +1Code Generation Repository-Level Code Generation
McGill University · Montreal, Quebec, Canada · University of Ottawa +1
Improving LLM Code Generation via Requirement-Aware Curriculum Reinforcement Learning
May 1, 2026Shouyu Yin, Zhao Tian, Junjie Chen +1RL for Code Generation Curriculum Learning
Tianjin University, Tianjin, China · Dalian Maritime University, Dalian, China
ScaleBox: Enabling High-Fidelity and Scalable Code Verification for Large Language Models
Apr 30, 2026Jiasheng Zheng, Xin Zheng, Boxi Cao +8LLM Code Generation
1Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences · 1Chinese Information Processing Laboratory Institute of Software, Chinese Academy of Sciences 2University of Chinese Academy of Sciences
To Diff or Not to Diff? Structure-Aware and Adaptive Output Formats for Efficient LLM-based Code Editing
Apr 30, 2026Wei Cheng, Yongchang Cao, Chen Shen +4Code Generation LLM Inference Acceleration
State Key Laboratory for Novel Software Technology, Nanjing University, China · Tongyi Lab, Alibaba Group, China

May 19, 2026, cs.AIJ/K move · Enter open · S save

PRISM: A Benchmark for Programmatic Spatial-Temporal Reasoning

Qiran Zhang, Yuheng Wang, Runde Yang, Lin Wu+8

School of Artificial Intelligence, Shanghai Jiao Tong University

Spatial Reasoning Code Generation Benchmarks LLM Code Generation Video Generation Evaluation

Open paper AskPDF arXiv

Programmatic video generation through code offers geometric precision and temporal coherence beyond pixel-level diffusion models, yet rigorously evaluating whether language models can produce spatially correct animated outputs remains an open problem. We introduce PRISM, a large-scale benchmark of 10,372 human-calibrated instruction-code pairs (20 times larger than prior programmatic video generation benchmarks), grounded in real-world knowledge visualization scenarios across English and Chinese and spanning 437 subject categories. We further propose a funnel-style evaluation framework with four complementary metrics: Code-Level Reliability for executability, Spatial Reasoning for layout correctness over full animation sequences, and Prompt-Aware Dynamic Visual Complexity (PADVC) and Temporal Density (TD) for diagnosing dynamic expression and temporal activity. Systematic evaluation of seven mainstream LLMs reveals a striking Execution-Spatial Gap: the average drop from execution success rate to spatial pass rate is approximately 41%, showing that runnable code does not necessarily yield spatially coherent visual output. These findings show that programmatic video generation evaluation should go beyond executability. PRISM provides a principled benchmark for advancing spatially coherent code generation.

Momentum

Related topics

Latest papers 196