LLMs for Cybersecurity

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

19 papers in the last four weeks, up 280% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 166

All topics

Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps
Apr 21, 2026Alankrit Chona, Igor Kozlov, Ambuj KumarAI Agent Security Benchmarks LLMs for Cybersecurity
Simbian AI
Evaluating LLM-Generated Obfuscated XSS Payloads for Machine Learning-Based Detection
Apr 21, 2026Divyesh Gabbireddy, Suman SahaSoftware Vulnerability Detection Adversarial Examples
Department of Computer Science and Engineering · The Pennsylvania State University · University Park, PA, USA
Do Agents Dream of Root Shells? Partial-Credit Evaluation of LLM Agents in Capture the Flag Challenges
Apr 21, 2026Ali Al-Kaswan, Maksim Plotnikov, Maxim Hájek +3LLM Agent Evaluation AI Agent Security Benchmarks
Delft University of Technology Delft, The Netherlands
Refute-or-Promote: An Adversarial Stage-Gated Multi-Agent Review Methodology for High-Precision LLM-Assisted Defect Discovery
Apr 21, 2026Abhinav AgarwalSoftware Vulnerability Detection LLMs for Cybersecurity
Systematic Capability Benchmarking of Frontier Large Language Models for Offensive Cyber Tasks
Apr 18, 2026Tyler H. Merves, Michael H. Conaway, Joseph M. Escobar +2LLM Agent Evaluation Cybersecurity
Department of Cybersecurity · Department of Information Sciences and Technology
Analyzing Chain of Thought (CoT) Approaches in Control Flow Code Deobfuscation Tasks
Apr 16, 2026Seyedreza Mohseni, Sarvesh Baskar, Edward Raff +1Software Reverse Engineering CoT Reasoning
University of Maryland Baltimore County, Baltimore, USA · CrowdStrike, New York, USA
Quantifying Frontier LLM Capabilities for Container Sandbox Escape
Mar 1, 2026Rahul Marchand, Art O Cathain, Jerome Wynne +5LLM Safety Benchmarks LLM Agent Security
University of Oxford · UK AI Security Institute
TrojanGYM: A Detector-in-the-Loop LLM for Adaptive RTL Hardware Trojan Insertion
Jan 23, 2026Saideep Sreekumar, Zeng Wang, Akashdeep Saha +6Backdoor Attacks Adversarial Robustness
NYU Abu Dhabi, Abu Dhabi, UAE · NYU Tandon School of Engineering, New York, USA
From Lab to Reality: A Practical Evaluation of Deep Learning Models and LLMs for Vulnerability Detection
Dec 11, 2025Chaomeng Lu, Bert LagaisseSoftware Vulnerability Detection LLMs for Cybersecurity
DistriNet Group-T, KU Leuven · Leuven, Belgium
Toward Cybersecurity-Expert Small Language Models
Oct 15, 2025Matan Levi, Daniel Ohayon, Ariel Blobstein +3Cyber Threat Intelligence Cybersecurity
1IBM Research.
What's on My Network? Using Large Language Models to Identify Real-World IoT Devices at Scale
Sep 24, 2025Rameen Mahmood, Tousif Ahmed, Sai Teja Peddinti +1Internet of Things LLMs for Cybersecurity
New York University, New York, NY, USA · Google, Mountain View, CA, USA
FALCON: Transforming Cyber Threat Intelligence into Deployable IDS Rules with Self-Reflection
Aug 26, 2025Shaswata Mitra, Subash Neupane, Martin Duclos +5Cyber Threat Intelligence Network Intrusion Detection
The University of Alabama · Meharry Medical College · The University of Texas at El Paso
Less Data, More Security: Advancing Cybersecurity LLMs Specialization via Resource-Efficient Domain-Adaptive Continuous Pre-training with Minimal Tokens
Jun 30, 2025Salahuddin Salahuddin, Ahmed Hussain, Jussi Löppönen +1Language Model Pretraining Domain-Adaptive Pretraining
SSH Communications Security, Helsinki, Finland · Aalto University, Espoo, Finland · Networked Systems Security (NSS) Group, KTH Royal Institute of Technology, Stockholm, Sweden
DMind Benchmark: Toward a Holistic Assessment of LLM Capabilities across the Web3 Domain
Apr 18, 2025Enhao Huang, Pengyu Sun, Shuxun Wang +13LLM Evaluation Software Vulnerability Detection
Zhejiang University Hangzhou, China · The State Key Laboratory of Blockchain and Data Security · DMind.ai Singapore +1
Large Language Models for Cryptocurrency Transaction Analysis: A Bitcoin Case Study
Jan 30, 2025Yuchen Lei, Yuexin Xiang, Rafael Dowsley +5LLM Evaluation LLMs for Cybersecurity
School of Cyber Science and Engineering, Wuhan University, Wuhan 430072, China · Faculty of Information Technology, Monash University, Clayton, VIC 3800, Australia · CSIRO’s Data61, Eveleigh, NSW 2015, Australia +2
ANVIL: Anomaly-based Vulnerability Identification without Labelled Training Data
Aug 28, 2024Weizhou Wang, Eric Liu, Xiangyu Guo +3Software Vulnerability Detection LLMs for Cybersecurity
University of Toronto, Toronto, Canada

Apr 21, 2026, cs.CRJ/K move · Enter open · S save

Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps

Alankrit Chona, Igor Kozlov, Ambuj Kumar

Simbian AI

AI Agent Security Benchmarks LLMs for Cybersecurity

Open paper AskPDF arXiv

We introduce the Cyber Defense Benchmark, a benchmark for measuring how well large language model (LLM) agents perform the core SOC analyst task of threat hunting: given a database of raw Windows event logs with no guided questions or hints, identify the exact timestamps of malicious events. The benchmark wraps 106 real attack procedures from the OTRF Security-Datasets corpus - spanning 86 MITRE ATT&CK sub-techniques across 12 tactics - into a Gymnasium reinforcement-learning environment. Each episode presents the agent with an in-memory SQLite database of 75,000-135,000 log records produced by a deterministic campaign simulator that time-shifts and entity-obfuscates the raw recordings. The agent must iteratively submit SQL queries to discover malicious event timestamps and explicitly flag them, scored CTF-style against Sigma-rule-derived ground truth. Evaluating five frontier models - Claude Opus 4.6, GPT-5, Gemini 3.1 Pro, Kimi K2.5, and Gemini 3 Flash - on 26 campaigns covering 105 of 106 procedures, we find that all models fail dramatically: the best model (Claude Opus 4.6) submits correct flags for only 3.8% of malicious events on average, and no run across any model ever finds all flags. We define a passing score as >= 50% recall on every ATT&CK tactic - the minimum bar for unsupervised SOC deployment. No model passes: the leader clears this bar on 5 of 13 tactics and the remaining four on zero. These results suggest that current LLMs are poorly suited for open-ended, evidence-driven threat hunting despite strong performance on curated Q&A security benchmarks.

Momentum

Related topics

Latest papers 166