Code Language Models

Follow this topic Ask about this topic

Code language models are studied for tasks including code completion, program understanding, debugging, translation, and repository-level software engineering. Common approaches combine code pretraining and instruction tuning with fill-in-the-middle training, retrieval, and feedback from execution, tests, or reinforcement learning. Evaluation ranges from unit-test benchmarks such as HumanEval to repository repair tasks such as SWE-bench, alongside multilingual, security, and robustness tests. A central challenge is that benchmark success does not always predict reliable behavior on complex code, unfamiliar instructions, or real-world security risks.

Latest papers

115 papers10 papers in the last four weeks, down 9% on the four weeks before. 0.1% of all new papers.

Software Engineering LLMs: Why We Need Them and How Software Engineering Helps Build Them
Oct 24, 2026Amir M. Ebrahimi, Mohammed Mehedi Hasan, Seyed Shayan Daneshvar +3Code Language Models LLM Training
Queen’s University, Canada · University of Manitoba, Canada · Center for Software Excellence, Huawei Technologies, Canada
Do LMs Know What Variables Do? Interpreting Code Representations via Variable Roles
Oct 16, 2026Célian Vasson, Daniil Ognev, Kentaro Inui +1LLM Interpretability Code Language Models
Sorbonne Université · MBZUAI · RIKEN AIP +1
Causal Syntax Circuits in CodeLMs: Budget and Probe Decide What Looks Localised
Oct 14, 2026Harsh Kumar, Tanmay Shailesh Joshi, Rahul Maity +1Circuits in Language Models Code Language Models
National Institute of Technology Agartala
LintEval: Benchmarking LLMs on Linter-Guided File Refactoring
Oct 14, 2026Tongwei Deng, Jinyu Zhu, Yue Cao +3Automated Code Refactoring Code Language Models
School of Computer Science, Peking University · Key Lab of HCST (PKU), MOE; SCS, Peking University · School of Software Engineering, Beijing Jiaotong University
DivAlgo: A systematic evaluation of algorithmic diversity for code generation models
Oct 13, 2026Anubhav Jangra, Smaranda MuresanCode Language Models Code Generation Evaluation
Department of Computer Science Columbia University, USA · Barnard College Columbia University, USA
SocraticRepair: Planning-Guided Socratic Tutoring for Program Debugging and Misconception Repair
Oct 10, 2026Mykhailo Kopytskyi, Jianhua Yang, Mir SeyedebrahimiSocratic Tutoring Intelligent Tutoring Systems
University of Warwick · Warwick Manufacturing Group
Mind the Gaps: From Failure Attribution to Closed-Form Repair of Code Language Models
Oct 4, 2026Jian Gu, Hongyu Zhang, Chunyang Chen +1Causal Interventions in Language Models Code Language Models
Monash University, Australia · Chongqing University, China · Technical University of Munich, Germany
Hidden in the Comments: A Context-Injection Attack Surface in Code LLMs
Oct 4, 2026Noor Munir, Francesco Quinzan, Stephen RobertsLLM Security Indirect Prompt Injection
Department of Engineering Science, University of Oxford, Oxford OX1 3PJ, UK
Complexity-Aware Evaluation of LLM Comprehension
Sep 29, 2026Ali Mohammadi Esfahani, Nafiseh Kahani, Samuel A. AjilaSoftware Engineering LLM Evaluation
Systems and computer Engineering Carleton University Ottawa, Canada
UNBIND: UNlearning By INference-time Directional Steering for Code LLMs
Sep 28, 2026Zhengyang Shan, Jiayun Xin, Yanjun Lin +7Activation Steering LLM Unlearning
Department of Computer Science and Technology Shandong University Qingdao, China
Style, Not Self: Surface Cues Explain Zero-Shot Code Attribution by Large Language Models
Sep 24, 2026Ehsan Barkhordar, Surendrabikram ThapaLanguage Model Fingerprinting Code Language Models
Koç University · Virginia Tech
On the Lexical Superstition of Large Language Models for Code Comprehension: Re-evaluation on Code of Low Lexical Quality
Sep 22, 2026Xin Shen, San-Zhuo Xi, Yali Du +1Language Model Robustness Code Language Models
Nanjing University, Nanjing, China
Rethinking Pivot Programming Languages in Code Language Models
Sep 19, 2026Andor Diera, Lukas Galke Poech, Matthias TichyCode Language Models Cross-Lingual Transfer
Ulm University Ulm, Germany · University of Southern Denmark Odense, Denmark
An Empirical Evaluation of Cost-Efficient Large Language Models on Algorithmic Programming Tasks
Sep 16, 2026Chandimal Adikari, Nandika HerathCode Generation LLM Reliability
Faculty of Engineering and Information Sciences University of Wollongong Wollongong, Australia · Faculty of Engineering, Computing and Science Western Sydney University Parramatta, Australia
Retrieval-Augmented Generation for Scientific Code Understanding
Sep 10, 2026Aaron Nobile, Andreas Adelmann, Mohsen SadrRetrieval-Augmented Generation LLM Grounding
Department of Physics, ETH Zürich, 8093 Zürich, Switzerland · Paul Scherrer Institute, 5232 Villigen PSI, Switzerland · Massachusetts Institute of Technology, Cambridge, MA 02139, USA
The Vibe Shift in Software Engineering: Evaluating AI-Led Conversational Programming for Performance, Cognition, and Responsible Adoption
Sep 9, 2026Sales G. Aribe, Louie Jay S. LabastidaCode Generation Evaluation Human-AI Collaboration
Information Technology Department, Bukidnon State University, Malaybalay City, Philippines
CodeTD: Topology of Attention Detects Hallucinations in Code LLMs
Sep 7, 2026Daria Voronkova, Ilya Trofimov, Anton Dmitriev +3LLM Hallucination Detection Code Generation Evaluation
Applied AI Institute · AXXX · IMJ CNRS
Diffs vs. Whole Files: An Empirical Comparison of Iterative Edit-Based and Direct Generation for Flutter/Dart Code Models
Sep 5, 2026Andrej AndrejevCode Language Models Code Generation
Independent researcher (bbidpa)
CROCODIL: Cross-Model Code Editing with LLMs
Sep 3, 2026Linghan Zhong, Aditya Thimmaiah, Jayanth Srinivasa +2Reinforcement Learning LLM Post-Training
1The University of Texas at Austin · 2Cisco Research
Hints Help But Do They Teach? Evaluating Skills Transfer in Code Generation
Sep 1, 2026Will BadrLanguage Model Generation Evaluation LLM Prompting
Predicting Program Exit Code with LLMs and Programming Language Semantics
Sep 1, 2026Lara Marinov, Aditya Thimmaiah, Jayanth Srinivasa +2LLM Evaluation Code Language Models
The University of Texas at Austin · Cisco Research
Evaluating Language Models on Cross-Language Code Functional Equivalence
Aug 25, 2026Hui Sun, Anderson Uchôa, Rohit Gheyi +1Code Language Models LLM Evaluation
Federal University of Campina Grande, Campina Grande, Paraíba, Brazil
Names Can Hurt: Spotting Slopsquatting Risks Caused by Package Name Hallucinations in Local Coding LLMs
Aug 24, 2026Akash Raj, Sargam SahuSoftware Supply Chain Security Code Language Models
M.Tech, School of Artificial Intelligence and Data Science Indian Institute of Technology Jodhpur · B.Tech, CSE Department Jaypee University of Engineering and Technology, Guna
Memorization Diagnostics for Code LLMs Should be Scale-Aware
Aug 13, 2026Prateek Kumar Rajput, Abdoul Aziz Bonkoungou, Alberick Euraste Djiré +6Memorization in Language Models Code Language Models
University of Luxembourg, Esch-sur-Alzette, Luxembourg.
Token Signatures of Code: Comparing Coding Behaviors Across Large Language Models
Aug 12, 2026Junpeng Wang, Yuzhong Chen, Menghai Pan +2Code Language Models LLM Evaluation
Visa Research
On the Robustness of LLMs' Internal Representation of Code Correctness
Aug 8, 2026Francisco Ribeiro, Sohaila Abdulsattar, Renata Gonzalez +2LLM Reliability Language Model Robustness
New York University Abu Dhabi Abu Dhabi, United Arab Emirates
LangChoiceBench: Measuring and Explaining Programming-Language Choice in LLMs
Aug 6, 2026Lukas Twist, Twm Stone, Helen Yannakoudakis +1LLM Evaluation Language Model Bias Evaluation
King’s College London, London, UK · MATS Research
OctoLong: Mid-Training On Cross-Repository Code Contexts Enhances Long-Context Modeling
Aug 5, 2026Indraneil Paul, Falko Helm, Goran Glavaš +1Software Engineering Long-Context Language Modeling
UKP Lab & Center for Applied Cybersecurity ATHENE, TU Darmstadt · CAIDAS, University of Würzburg
To Add Is Machine, To Delete Is Human: Measuring and Mitigating Deletion Avoidance in LLM Code Editing
Jul 30, 2026Amir M. Ebrahimi, Mohammed Mehedi Hasan, Aaditya Bhatia +2LLM Reliability Language Model Editing
School of Computing, Queen’s University, Kingston, Ontario, Canada
Learning from 53.6K Real-World Developer Edits of AI-Generated Code
Jul 27, 2026Jenny T. Liang, Mihika Bairathi, Wayne Chi +3Code Generation Evaluation Code Language Models
Carnegie Mellon University

Oct 24, 2026J/K move · Enter open · S save

Software Engineering LLMs: Why We Need Them and How Software Engineering Helps Build Them

Amir M. Ebrahimi, Mohammed Mehedi Hasan, Seyed Shayan Daneshvar, Jiahuei Lin+2

Queen’s University, Canada · University of Manitoba, Canada · Center for Software Excellence, Huawei Technologies, Canada

Code Language Models LLM Training LLM Code Generation Automated Software Engineering

Open paper AskPDF arXiv

Software engineering (SE) expertise can shape how coding large language models (LLMs) are trained and which engineering capabilities they acquire. We examine this role through an industrial case study of capability-focused training for a 92B-parameter coding LLM serving more than 100,000 developers. We demonstrate how established SE knowledge can guide the selection of engineering capabilities, the construction of verifiable training data, and the evaluation of model improvements. To target repository-level refactoring, we developed RefactorForge, a pipeline that combines refactoring knowledge, established SE tools, and verification techniques to construct training data. Across 306 Python repositories, RefactorForge generated 5,704 executable tasks and retained 4,572 verified teacher trajectories totaling 72.51 million tokens. Incremental supervised fine-tuning on these trajectories, accounting for 0.46% of the preceding coding mixture's tokens, improved RefactorBench performance by 18.00 percentage points and SWE-Bench ProMax performance by 7.06 points. Under OpenCode, these improvements extended to SWE issue resolution, with gains of 4.97 points on SWE-Bench Verified and 1.23 points on SWE-Bench Pro. Improvements also persisted across other coding-agent harnesses, while greenfield repository generation showed mixed results. We report lessons from constructing capability-focused training data and evaluating its effects across engineering workloads. Building on this industrial experience, we propose a research agenda for expanding SE's role in transforming coding LLMs into SE LLMs: incorporating SE knowledge into model training, applying engineering methods to model development and maintenance, and systematically defining the engineering capabilities these models should acquire.