AI Slop and Hallucinations in Vulnerability Assessment: A Survey on Reasoning Failures and Trustworthy Mitigation
Organizations: University of New South Wales, High St, Kensington NSW 2052, Australia · University of Wollongong, Northfields Ave, Wollongong NSW 2500 Australia · Nanjing University of Information Science & Technology, 219 Ningliu Road, Nanjing, Jiangsu, China · Griffith University, Parklands Drive, Southport QLD 4222 Australia ·
Abstract
The integration of Large Language Models (LLMs) into cybersecurity has transformed vulnerability assessment, but it has also produced a trustworthiness crisis driven by the unchecked proliferation of "AI slop." These artifacts, hallucinated vulnerabilities, plausible but incorrect patches, and semantically repackaged bug reports, impose a cognitive burden on human triage pipelines that mirrors a denial-of-service attack. This paper surveys the empirical evidence, identifies a unifying mechanism, and traces a path toward trustworthy triage. We formalize a taxonomy of AI slop grounded in a structured literature review and dissect its root cause: the gap between the causal deductive reasoning of security experts and the autoregressive probabilistic generation of current LLMs. We operationalize this gap through a measurable proxy, the Deductive Coverage Score, and show that chain-of-thought prompting and tool-using agents narrow but do not close it. We review mitigation strategies and argue that passive detection and watermarking target provenance rather than correctness, facing fundamental entropy constraints. We instead advocate for active neuro-symbolic verification, mapping each pipeline component to prior systems with documented limits on security inputs. Finally, we specify two evaluation instruments, CVE-Bench and Slop-Score, including dataset construction, metric formulas, and anti-gaming provisions. By shifting evaluation from linguistic fluency to mathematical verifiability, this survey provides a roadmap for securing emerging AI-driven triage systems.
Figures & tables
| Reference | Cat. | Failure Subtype | Key Reasoning Failure | Grounding |
|---|---|---|---|---|
| Ullah et al. [ 31 ] | H | Sycophantic halluc. | Hallucinates vulns in safe code to satisfy prompts | None |
| Zhang et al. [ 38 ] | H | Token-bias confound. | Judgment shifts with surface renaming, not logic | None |
| Chakraborty et al. [ 4 ] | H | Spurious feature learn. | Learns syntactic artifacts, not exec. semantics | Partial |
| Pearce et al. [ 22 ] | H | Tool-use confab. | Agents misinterpret tool output in security tasks | Tool logs |
| Pearce et al. [ 23 ] | IP | Symptom-only repair | Fixes surface, misses root cause, adds regressions | Compile |
| Xia et al. [ 36 ] | IP | Repair loop drift | Plausible fixes silently break arch. state | Tests |
| Benchmark | Focus | Security | Gap for AI Slop |
| HumanEval / MBPP | Standalone code synthesis | None | Binary pass/fail; no security context |
| SWE-bench [ 12 ] | Repo-level patching | Partial | Evaluates regression, ignores deceptive fluency |
| CyberSecEval [ 2 ] | Compliance & insecure coding | High | Tests jailbreaks, not benign-code hallucination |
| CVE-Bench | Phantom exploit detection | High | Isolates deceptive generation with execution-validated ground truth |
| Slop-Score | Fluency vs. substance | High | Continuous metric penalizing the fluency-substance gap |
LLMs for Cybersecurity
Large language models for cybersecurity support threat analysis, vulnerability research, security operations, and cyber defense.
See all 258 papers