Overview

Code language models are studied for tasks including code completion, program understanding, debugging, translation, and repository-level software engineering. Common approaches combine code pretraining and instruction tuning with fill-in-the-middle training, retrieval, and feedback from execution, tests, or reinforcement learning. Evaluation ranges from unit-test benchmarks such as HumanEval to repository repair tasks such as SWE-bench, alongside multilingual, security, and robustness tests. A central challenge is that benchmark success does not always predict reliable behavior on complex code, unfamiliar instructions, or real-world security risks.

Latest papers

CardsList

125 papers7 papers in the last four weeks, down 36% on the four weeks before. 0.1% of all new papers.

  1. RedCoder: Automated Multi-Turn Red Teaming for Code LLMs

    Jun 25, 2025Wenjie Jacky Mo, Qin Liu, Xiaofei Wen +5Adversarial Prompt GenerationLLM Red Teaming

  2. LLMs are Brittle to Simple Code Transformations: Introducing CETBench - A Benchmark for Code-Equivalence Checking

    Jun 4, 2025Neeva Oza, Ishaan Govil, Parul Gupta +3LLM EvaluationLanguage Model Robustness

  3. Is Your Benchmark Still Useful? Dynamic Benchmarking for Code Language Models

    Mar 9, 2025Batu Guan, Xiao Wu, Yuanyuan Yuan +1Benchmark DesignData Contamination in Language Models

  4. Astraios: Parameter-Efficient Instruction Tuning Code Large Language Models

    Jan 1, 2024Terry Yue Zhuo, Armel Zebaze, Nitchakarn Suppattarachai +4LLM Fine-TuningInstruction Tuning

  5. On Robustness of Prompt-based Semantic Parsing with Large Pre-trained Language Model: An Empirical Study on Codex

    Jan 30, 2023Terry Yue Zhuo, Zhuang Li, Yujin Huang +4Adversarial Robustness of Language ModelsSemantic Parsing