cs.CRNov 4, 2026

A.S.E 2.0: Understanding How Repository Context Impacts the Security of LLM-Generated Code

Authors: Jiongchi Yu, Keke Lian, Junqing Wang, Zhikang Fan, Zhaoxuan Li, Yongheng Huang, Ziming Zhao, Dong Zhang

Organizations: Nanyang Technological University Singapore, Singapore · Tencent China · Zhejiang University Hangzhou, China · IIE, CAS Beijing, China

Abstract

Large language model (LLM) coding assistants and agents construct a repository context before generating code. For security-sensitive code, this context decides whether the model sees the project's safeguards (e.g., authorization checks) or insecure code to imitate. However, context construction is designed almost exclusively for functional correctness, and secure-code benchmarks treat the context as fixed, so how repository context affects the security of generated code remains unclear. We construct CASE-Bench, a benchmark of 145 repository-level tasks from real CVEs in 51 projects. Each task runs in Docker, has functional tests and a proof-of-concept exploit validated in both directions, and is annotated with the repository code that enforces or undermines its security. Using CASE-Bench, we compare 13 context construction methods and six controls with six LLMs, edit single context elements, and analyze coding-agent trajectories. We find that (1) functionality-oriented context raises the functional pass rate by up to 8.2 percentage points (pp) but the secure pass rate by at most 3.7 pp, (2) withholding a safeguard lowers the secure pass rate by up to 8.1 pp, while one inserted insecure exemplar raises correct-but-exploitable code by 6.3 pp, and (3) coding agents observe the relevant safeguard in only 27-36% of trajectories. These findings inform security-aware context construction and secure-code evaluation.

Figures & tables

Explore similar work

CardsList
  1. SecCodeBench-V2 Technical Report

    Feb 17, 2026Longfei Chen, Ji Zhao, Lanxiao Cui +24Secure Code GenerationCode Generation Benchmarks

  2. Poster: Rethinking Security in LLM Code Generation through Real-World Risk Scenarios

    Jul 25, 2026Lixun Ma, Ruolong Ma, Bei Wang +4LLM Safety BenchmarksLLM Security

  3. Enhancing Reliability in LLM-Based Secure Code Generation

    May 22, 2026Mohammed F. Kharma, Mohammad Alkhanafseh, Ahmed Sabbah +1LLM EvaluationCoT Reasoning