Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit

Souradip Nath; Chih-Yi Huang; Aditi Ganapathi; Kashyap Thimmaraju; Jaron Mink; Gail-Joon Ahn

cs.CRApr 11, 2026

Like a Hammer, It Can Build, It Can Break: Large Language Model Uses, Perceptions, and Adoption in Cybersecurity Operations on Reddit

Authors: Souradip Nath, Chih-Yi Huang, Aditi Ganapathi, Kashyap Thimmaraju, Jaron Mink, Gail-Joon Ahn

Organizations: Arizona State University · Technische Universität Berlin

Abstract

Large language models (LLMs) have recently emerged as promising tools for augmenting Security Operations Center (SOC) workflows, with vendors increasingly marketing autonomous AI solutions for SOCs. However, there remains a limited empirical understanding of how such tools are used, perceived, and adopted by real-world security practitioners. To address this gap, we conduct a mixed-methods analysis of discussions in cybersecurity-focused forums to learn how a diverse group of practitioners use and perceive modern LLM tools for security operations. More specifically, we analyzed 892 posts between December 2022 and September 2025 from three cybersecurity-focused forums on Reddit, and, using a combination of qualitative coding and statistical analysis, examined how security practitioners discuss LLM tools across three dimensions: (1) their stated tools and use cases, (2) the perceived pros and cons of each tool across a set of critical factors, and (3) their adoption of such tools and the expected impacts on the cybersecurity industry and individual analysts. Overall, our findings reveal nuanced patterns in LLM tools adoption, highlighting independent use of LLMs for low-risk, productivity-oriented tasks, alongside active interest around enterprise-grade, security-focused LLM platforms. Although practitioners report meaningful gains in efficiency and effectiveness in LLM-assisted workflows, persistent issues with reliability, verification overheads, and security risks sharply constrain the autonomy granted to LLM tools. Based on these results, we also provide recommendations for developing and adopting LLM tools to ensure the security of organizations and the safety of cybersecurity practitioners.

Figures & tables

Subreddits	Threads (Pulled)	Threads (Relevant)	Posts (Pulled)	Posts (Relevant)
r/cybersecurity	830	69 (8.31%)	1,665	863 (51.83%)
r/Information_Security	96	5 (5.21%)	21	15 (71.43%)
r/ciso	25	2 (8.00%)	17	14 (82.35%)
r/blueteamsec	187	0	–	–
r/CyberSecurityAdvice	76	0	–	–
r/cybersecurity_help	36	0	–	–

Table 1: Collected Reddit Dataset – number of threads retrieved per subreddit, number and share of relevant threads, and the number (and share) of posts/comments coded as relevant.

Categories of LLM Tools	$\#$ (Out of 410)
By Purpose
General-Purpose LLM Tools	248 (60.49%)
Security-Specific LLM Tools	180 (43.90%)
By Origin
Commercially Available	398 (97.07%)
In-House Built	14 (3.42%)

Table 2: Reported LLM Tools – Categorized by purpose and origin. Note that individual posts may mention multiple tools.

Types of LLM Use Cases	Description	# General	# Security	$p$ -value
Triage & Incident Response ( $n$ = $139$ )	Support for alert triage, investigation, correlation, mitigation, and response workflows.	30 (40.54%)	44 (59.46%)	< 0.001*
Scripting & Query Support ( $n$ = $88$ )	Generation and refinement of security scripts and detection queries.	51 (89.47%)	6 (10.53%)	< 0.01*
Reporting & Documentation ( $n$ = $84$ )	Drafting and summarizing investigation reports, policies, and threat briefings, etc.	43 (89.58%)	5 (10.42%)	< 0.05*
Threat Analysis ( $n$ = $54$ )	Support for risk and vulnerability analysis, threat hunting, and modeling workflows.	16 (57.14%)	12 (42.86%)	0.948
Knowledge Support ( $n$ = $53$ )	Concept explanation, information retrieval, document-based knowledge extraction.	31 (88.57%)	4 (11.43%)	0.119
Training, Compliance, Others ( $n$ = $18$ )	Drafting training exercises, reviewing compliance materials, and other non-core tasks.	9 (81.82%)	2 (18.18%)	–

Table 3: Reported LLM Tools Use Cases and Descriptions Across Tool Categories (General-Purpose, Security-Specific).

Factors	Description	# Positive	# Negative	$p$ -value
Capabilities	Specific tasks and SOC use cases where LLM tools are perceived as helpful	174 (64.21%)	97 (35.79%)	< 0.001*
Efficiency	Time-related aspects, such as speed of analysis, deployment, and workload reduction	54 (84.38%)	10 (15.62%)	< 0.001*
Reliability	Trustworthiness, accuracy, and consistency of LLM-generated insights	3 (5.17%)	55 (94.83%)	< 0.001*
Security & Privacy	Security of LLM tools, organizational data governance, and privacy considerations	6 (11.54%)	46 (88.46%)	< 0.001*
Autonomy	Ability of LLM tools to operate independently with minimal human supervision	6 (12.77%)	41 (87.23%)	< 0.001*
Cost	Financial aspects, such as operational costs and perceived return on investment	5 (14.29%)	30 (85.71%)	< 0.001*

Table 4: Reported LLM Factors, Descriptions and Opinions.

	Counts			p1 vs p2		p2 vs p3
Adoption Stage	p1	p2	p3	Delta	$p$ -val	Delta	$p$ -val
Evaluating	20	16	87	+4.40%	1.000	-33.65%	0.002*
Using	12	7	181	-7.36%	1.000	+31.09%	0.013*
Not Using	3	3	44	+2.97%	1.000	+2.56%	1.000

Table 5: Reported Adoption of LLM Tools Over Time.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 1 — Figure 1: Number of Threads in Our Dataset Over Time.

Topics of Discussion	# Total Posts (Out of 892)
LLM Uses (§ 4 )
Tools Mentioned	410 (45.96%)
Use Cases Mentioned	325 (36.44%)
Perceptions of LLM Tools (§ 5 )
Opinions Shared	406 (45.52%)
LLM Factors	459 (51.46%)

Appendix

Table 6: Distribution of Discussion Topics – Identified across 892 relevant posts. Individual posts often addressed multiple themes.

Tool Name	Freq.	Description
General-Purpose LLM Tools
ChatGPT	90	A general-purpose conversational LLM developed by OpenAI
MS Copilot	30	A generative AI assistant integrated across Microsoft products
Claude	9	A conversational LLM developed by Anthropic
Gemini	6	A multimodal LLM developed by Google
Llama	5	A family of open-source large language models released by Meta

Appendix

Table 7: Comprehensive List of Reported LLM Tools – grouped by general-purpose and security-specific platforms, along with their observed frequencies.

LLM Tools	# Positive	# Negative
General-Purpose Tools	88 (50.57%)	86 (49.43%)
Unnamed (LLMs, GenAI, etc.)	36 (42.35%)	49 (57.65%)
ChatGPT and OpenAI models	38 (58.46%)	27 (41.54%)
Microsoft Copilot	10 (50.00%)	10 (50.00%)
Claude	4 (66.67%)	2 (33.33%)
Gemini	0 (0.00%)	5 (100.00%)

Appendix

Table 8: Practitioner Sentiment Across LLM Tools.

LLMs for Cybersecurity

Large language models for cybersecurity support threat analysis, vulnerability research, security operations, and cyber defense.

See all 255 papers

From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness
Aug 1, 2026Jonas Thurner, Nadine Jost, Stefan Albert Horstmann +4Cybersecurity LLMs for Cybersecurity
FH Münster, Germany · Ruhr University Bochum, Germany · University of Cologne, Germany +2
Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making
Aug 21, 2026Pasquale Malacaria, Yunxiao ZhangLLMs for Cybersecurity LLM Decision-Making
School of Electronic Engineering and Computer Science, Queen Mary University of London, London, UK. · Department of Computer Science, University of Exeter, Exeter, UK.
LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity
Jun 26, 2026Yiwei Xu, Yong Zhuang, Xuanming Liu +6Autonomous Cyber Defense LLM Agent Security
School of Cyber Science and Engineering, Wuhan University, Wuhan, China. · Department of Computer Science and Engineering, University at Buffalo, Buffalo, USA.

Aug 1, 2026J/K move · Enter open · S save

From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness

Jonas Thurner, Nadine Jost, Stefan Albert Horstmann, Fabian Ising+3

FH Münster, Germany · Ruhr University Bochum, Germany · University of Cologne, Germany+2

Cybersecurity LLMs for Cybersecurity

Open paper AskPDF arXiv

Security Operations Centers (SOCs) process large volumes of security events, requiring analysts to accurately detect and assess ongoing cyberattacks under time pressure. Recent advances in Large Language Models (LLMs) suggest potential benefits for security operations, yet their practical suitability for real-world SOC workflows remains poorly understood. To address this gap, we conducted 25 semi-structured interviews with SOC practitioners who had prior experience with LLMs, complemented by interactive scenarios to anticipate challenges and identify opportunities for the responsible integration of LLM-based tools into SOC workflows. We identified 15 LLM use cases grouped into six functional categories. While LLMs are valued for automating repetitive, low-level tasks such as report automation, practitioners rate high-impact tasks such as incident analysis as not yet feasible, reporting limitations in technical depth, context awareness, and organization-specific knowledge. They locate these limitations less in the models than in the readiness of their SOCs and human factors driving over-reliance. Despite concerns, practitioners express a strong willingness to adopt LLMs, describing competitive pressure that leaves few alternatives. This work contributes an empirical, practitioner-driven analysis of LLM use across SOC roles and organizations and derives concrete design and integration requirements for human-centered, operationally safe LLM-assisted security operations.

Abstract

LLMs for Cybersecurity

Explore similar work

From Chasing Ghosts to Missed Attacks: Perspectives and Perceptions of SOC Practitioners on LLM Integration, Risks, and Readiness

Structured but Fragile: On the Limits of LLMs in Cybersecurity Decision-Making

LLM agents security duality: a comprehensive survey of self-security and empowered cybersecurity