cs.AIAug 21, 2026

Large Language Models at the Intersection of Software Engineering and Software Security:An Evidence-Centered Structured Survey and Research Agenda

Authors: Wei Lin, Tao Zhou, Zhaofei Xie, Changgui Hong

Organizations: Nanjing Liancheng Intelligent Technology Group, Nanjing, China

Abstract

Large Language Models (LLMs) are moving from code completion toward repository-scale agents that retrieve context, edit files, execute tools, and participate in security-sensitive workflows. The evidence for these systems, however, remains divided between software engineering evaluations centered on functional task completion and software security evaluations centered on vulnerability detection, secure generation, or exploit-oriented validation. This evidence-centered structured survey synthesizes representative work available through May 31, 2026 across software engineering tasks, software security tasks, adaptation mechanisms, artifact granularity, and evaluation design. In addition to a task taxonomy, we introduce an assurance framework that separates functional correctness, security, operational reliability, evidence provenance, and agent authority. The review shows that execution feedback and repository access can substantially improve engineering task completion, but do not by themselves establish security; conversely, static-analysis labels or vulnerability-classification scores rarely establish deployable correctness. We identify recurring validity threats--weak test oracles, duplicated and temporally leaked data, changing agent harnesses, proxy-only security checks, and under-reported budgets and human intervention--and derive a minimum reporting protocol for cross-study comparison. The resulting research agenda prioritizes jointly secure-and-functional benchmarks, repository-scale threat models, calibrated human oversight, longitudinal maintainability evidence, and reproducible agent evaluation. The central conclusion is that model capability should be judged as an assurance case supported by task-appropriate evidence, rather than by a single benchmark score.

Figures & tables

Explore similar work

CardsList
  1. SEC-bench Pro: Can Language Models Solve Long-Horizon Software Security Tasks?

    May 26, 2026Hwiwon Lee, Jiawei Liu, Dongjun Kim +3Software Vulnerability DetectionSoftware Security

  2. LLM-Based Agents for Software and Systems Security: Approaches, Applications, and Assessment

    Aug 28, 2026Jingjing Nie, Jiawei Guo, Krishna Meda +1LLM Agent SecurityLLMs for Cybersecurity

  3. TOSSS: a CVE-based Software Security Benchmark for Large Language Models

    Mar 11, 2026Marc Damie, Murat Bilgehan Ertan, Domenico Essoussi +3LLM Safety BenchmarksSoftware Security