cs.CROct 22, 2026

Social-HUMINT: An LLM-Augmented Unified Framework for Measuring Social Media Exposure

Authors: Pellegrino Casoria, Simon Pietro Romano

Organizations: University of Naples Federico II, Department of Electrical Engineering and Information Technology (DIETI), Naples, Italy

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

In the post-API landscape, social-media profiling has become inexpensive and automated: Large Language Models (LLMs) can extract structured information from unstructured content and deduce attributes users never disclosed. A person's true exposure is not what they post but what can be inferred from it. As these practices become routine, exposure to social engineering attacks becomes an operational certainty, demanding a shared framework for assessment and mitigation. We make this exposure measurable by quantifying how accurately a target can be profiled, and how far that profile enables social engineering attacks. Because no existing model fully spans this continuum, we unify Social Media Intelligence (SOCMINT) and Cyber Human Intelligence (Cyber-HUMINT), two individual-focused disciplines that evolved separately, into a single doctrine-grounded taxonomy of 38 capabilities across the Cyber Threat Intelligence lifecycle, where user profiling bridges passive collection and active engagement. Domain experts validated the taxonomy (Fleiss' κκ = 0.92). We operationalize its core path (extraction, inference, and susceptibility assessment) as an LLM-augmented multi-agent pipeline, evaluated on real participants' public data. Susceptibility is assessed indirectly: the pipeline predicts how each target would rank personalized engagement options, matching their actual choices significantly above chance. This ranking capacity captures the personalization on which tailored attacks depend. The underlying profiles are accurate: extraction reaches F1 = 97.3%, and inferred attributes score 4.73/5, both far above rule-based and NER-based baselines at negligible cost. The framework augments security analysts' tradecraft rather than replacing it and enables defenders to gauge their own exposure before an adversary does.

Figures & tables

Figure 1: The SOCMINT and Cyber-HUMINT continuum: positioning of this work.
Threat ActorTypeCountryCampaignsSocial ProfilingHuman Engage.TargetImpactRef.
Contagious InterviewNation StateKPDEV#POPPERProfile Scraping Job/Role Enum.Fake Job OffersEmployees/InsidersFinancial Theft[ 8 ]
Lazarus GroupNation StateKPOp. Dream Job Op. In(ter)ceptionJob/Role Enum. People NetworkFake Job Offers Spear PhishingExecutives Employees/InsidersFinancial Theft Data Breach[ 5 ]
EXOTIC LILYCybercriminalRUBumblebee BazarLoaderProfile Scraping Job/Role Enum.Spear Phishing ImpersonationExecutives Employees/InsidersData Breach Service Disruption[ 41 ]
KimsukyNation StateKPBabyShark AppleSeedProfile Scraping Behavioral AnalysisImpersonationJournalists/ActivistsData Breach[ 42 ]
APT42Nation StateIRCharming Kitten SpearSpecterProfile Scraping Behavioral AnalysisImpersonation DisinformationPolitical FiguresElectoral Interf. Reputation Damage[ 43 ]
Scattered SpiderCybercriminalUS/UKMGM Breach Caesars BreachJob/Role Enum. People NetworkVoice PhishingIT/HelpdeskService Disruption Data Breach[ 44 ]
Table 1: Documented adversarial campaigns exploiting social-media intelligence.
PlatformUser VisibilityPlatform / API AccessAbuse ResilienceToS / Case Law
FacebookUser-controlled (default: Friends)Graph API; OAuth (user / app / page tokens); App ReviewEDM team; rate limits; behavioral analysis; client-side defenses; account banMeta v. Bright Data
LinkedInUser-controlled (default: Public); connection-degree visibilityOAuth (self only); Marketing / Talent API (no scraping)CASAL; behavioral analysis; content moderation; anti-scraping; ML real-time risk assessment; account banhiQ v. LinkedIn
InstagramUser-controlled (default: Public)Graph API; OAuth (Business / Creator only); App ReviewEDM team; rate limits; behavioral analysis; client-side defenses; account banMeta v. Bright Data
X/TwitterUser-controlled (default: Public)Paid tiers (Free/Basic/Pro); OAuth (app + user level)Rate limits; anti-spam; account banX v. Bright Data
TikTokUser-controlled (default: Public)Research API; Business API (no scraping); VCE (non-profit only)CAPTCHAs; rate limits; behavioral analysis; anti-scrapingEC v. TikTok
Table 2: Comparative summary of social media platform security constraints.
Figure 2: Social-HUMINT framework: capability classification across CTI phases.
Figure 3: LLM-augmented Social-HUMINT pipeline architecture.
Figure 4: Two-tier user profiling data model.
Planning & DirectionCollectionProcessingAnalysis & Corr.Dissemination
Strategy definition 4.9±0.34.9{\pm}0.3 [1.00]Passive observation 4.9±0.34.9{\pm}0.3 [1.00]Data triage & filtering 5.0±0.05.0{\pm}0.0 [1.00]Behavior & pattern 4.9±0.34.9{\pm}0.3 [1.00]Info sharing 4.8±0.44.8{\pm}0.4 [1.00]
Risk assessment 4.5±0.74.5{\pm}0.7 [0.92]Automated acquisition 4.9±0.34.9{\pm}0.3 [1.00]Multi-channel normaliz. 4.2±0.64.2{\pm}0.6 [0.92]Media analysis 4.1±0.54.1{\pm}0.5 [0.92]Threat reporting 4.9±0.34.9{\pm}0.3 [1.00]
Intel. requirements 4.8±0.64.8{\pm}0.6 [0.92]Active engagement 4.0±0.74.0{\pm}0.7 [0.77]KB ingestion 4.2±0.84.2{\pm}0.8 [0.77]People network 4.9±0.34.9{\pm}0.3 [1.00]Lessons learnt 4.2±0.64.2{\pm}0.6 [0.92]
Legal & ethical 4.8±0.64.8{\pm}0.6 [0.92]Multimedia collection 4.2±0.74.2{\pm}0.7 [0.85]Entity extraction 4.7±0.54.7{\pm}0.5 [1.00]Trend & temporal 4.4±0.74.4{\pm}0.7 [0.92]Dashboard & visual. 3.8±1.13.8{\pm}1.1 [0.69]
Collection strategy 4.7±0.64.7{\pm}0.6 [0.92]Crowd-sourced collect. 3.2±0.73.2{\pm}0.7 [0.23]Identity resolution 4.5±0.54.5{\pm}0.5 [1.00]Intent & deception 4.5±0.54.5{\pm}0.5 [1.00]Intel. liaison 4.1±0.84.1{\pm}0.8 [0.77]
Digital identity 4.2±0.64.2{\pm}0.6 [0.92]Historical research 4.0±0.64.0{\pm}0.6 [0.85]Data enrichment 4.3±0.54.3{\pm}0.5 [1.00]Debriefing 4.0±0.84.0{\pm}0.8 [0.69]Oper. actionability 4.8±0.44.8{\pm}0.4 [1.00]
Figure 5: Expert panel validation ( n=13n=13 ). Relevance expressed as μ±σ\mu{\pm}\sigma [I-CVI]. Scale: 1 (not relevant) to 5 (highly relevant). I-CVI ∈[0,1]\in[0,1] , I-CVI ≥0.78\geq 0.78 acceptable. →\rightarrow phase reassigned.
CategorynnTPFPFNPP %RR %F1F_{1} %
Identity14444091.710095.7
Experience142494098.410099.2
Education141842498.997.998.4
Certifications1212001610088.293.8
Projects42400100100100
Places Visited11695593.293.293.2
Table 3: Extraction performance by field category. nn = participants with at least one item.
  RatingCalibr.
ClassCategoryμ\muIQRσ\sigmaMinConfDiff
TargetShort Bio4.571.000.6534.47−- 0.10
Main Skills4.7900.5834.40−- 0.39
OpSec Awareness4.7900.5833.94−- 0.84
Tech. Sophisticat.4.8600.3644.15−- 0.71
Org. Access4.570.750.7634.19−- 0.38
Table 4: Inference agreement and confidence calibration ( n=14n=14 ). Rating: participant agreement (0–5). Conf: model confidence (0–5). Diff = Conf −- Rating; negative = underconfidence.
Figure 6: Engagement assessment: per-subject profiling accuracy. Diamonds: mean τ\tau ; dots: per-category means.
Figure 7: Engagement assessment: ranking quality distribution by question category.
Figure 8: Perception survey: likert-scale responses (0–5, n=14n=14 ). Bars extend left for ratings ≤2\leq 2 and right for ratings ≥3\geq 3 .
MethodTPFPFNPP %RR %F1F_{1} %
Regex17131556535.223.228.0
spaCy NER5364,9302009.872.817.3
LLM pipeline711152597.996.697.3
Table 5: Baseline comparison. Ground-truth: n=736n=736 items (TP+FN). Inferred items (252) excluded as unreachable by both baselines.
More in this topic

Social Media Analysis

Social media analysis studies online posts, media, users, and interactions to understand communication, communities, and public behavior.

See all 221 papers

Explore similar work

CardsList
Sep 11, 2026

GraphProfiler: Source-Linked Sensitive Attribute Inference via Personal Knowledge Graphs

Sensitive attributes such as age, income, and occupation can be inferred from user-generated content by aggregating indirect cues across many ordinary posts. LLM-based profilers can perform this aggregation automatically and with high accuracy, which makes large-scale personal attribute inference a major privacy threat. Existing LLM-based profilers, however, offer limited insight into which specific posts, concepts, and relationships made an inference possible, which is key to targeted privacy mitigation, i.e., redacting or rewriting only the few posts that actually leak an attribute, rather than perturbing entire histories. We introduce GraphProfiler, an auditable LLM-based profiler that represents each user's post history as a source-linked personal knowledge graph where nodes and edges trace back to the originating post and resolves attribute predictions to cited graph records and source texts. GraphProfiler reaches 86.7% attack success rate on the eight-attribute SynthPAI benchmark, within two points of strong text-only baselines, and 84.6% on PANDORA, while citing supporting evidence for over 98% of predictions. Our controlled ablation experiments provide evidence that the cited posts contribute to attack success, as removing them reduces the attack success rate substantially more than removing an equal number of random posts.

Ahmed Sohair Khan, Estrid He, Chenglong Ma +2

RMIT University, Melbourne, Australia

Privacy AuditingPrivacy Leakage in Language ModelsKnowledge GraphsUser Attribute Inference Attacks
AskPDF
Apr 21, 2026

Cyber Defense Benchmark: Agentic Threat Hunting Evaluation for LLMs in SecOps

We introduce the Cyber Defense Benchmark, a benchmark for measuring how well large language model (LLM) agents perform the core SOC analyst task of threat hunting: given a database of raw Windows event logs with no guided questions or hints, identify the exact timestamps of malicious events. The benchmark wraps 106 real attack procedures from the OTRF Security-Datasets corpus - spanning 86 MITRE ATT&CK sub-techniques across 12 tactics - into a Gymnasium reinforcement-learning environment. Each episode presents the agent with an in-memory SQLite database of 75,000-135,000 log records produced by a deterministic campaign simulator that time-shifts and entity-obfuscates the raw recordings. The agent must iteratively submit SQL queries to discover malicious event timestamps and explicitly flag them, scored CTF-style against Sigma-rule-derived ground truth. Evaluating five frontier models - Claude Opus 4.6, GPT-5, Gemini 3.1 Pro, Kimi K2.5, and Gemini 3 Flash - on 26 campaigns covering 105 of 106 procedures, we find that all models fail dramatically: the best model (Claude Opus 4.6) submits correct flags for only 3.8% of malicious events on average, and no run across any model ever finds all flags. We define a passing score as >= 50% recall on every ATT&CK tactic - the minimum bar for unsupervised SOC deployment. No model passes: the leader clears this bar on 5 of 13 tactics and the remaining four on zero. These results suggest that current LLMs are poorly suited for open-ended, evidence-driven threat hunting despite strong performance on curated Q&A security benchmarks.

Alankrit Chona, Igor Kozlov, Ambuj Kumar

Simbian AI

AI Agent Security BenchmarksLLMs for Cybersecurity
AskPDF
May 14, 2026

Talk is (Not) Cheap: A Taxonomy and Benchmark Coverage Audit for LLM Attacks

We introduce a reusable framework for auditing whether LLM attack benchmarks collectively cover the threat surface: a 4×\times6 Target ×\times Technique matrix grounded in STRIDE, constructed from a 507-leaf taxonomy -- 401 data-populated and 106 threat-model-derived leaves -- of inference-time attacks extracted from 932 arXiv security studies (2023--2026). The matrix enables benchmark-external validation -- auditing collective coverage rather than individual benchmark consistency. Applying it to six public benchmarks reveals that the three primary frameworks (HarmBench, InjecAgent, AgentDojo) occupy non-overlapping cells covering at most 25% of the matrix, while entire STRIDE threat categories (Service Disruption, Model Internals) lack any standardized evaluation, despite published attacks in these categories achieving 46×\times token amplification and 96% attack success rates through mechanisms which no benchmark tests. The corpus of 2,521 unique attack groups further reveals pervasive naming fragmentation (up to 29 surface forms for a single attack) and heavy concentration in Safety & Alignment Bypass, structural properties invisible at smaller scale. The taxonomy, attack records, and coverage mappings are released as extensible artifacts; as new benchmarks emerge, they can be mapped onto the same matrix, enabling the community to track whether evaluation gaps are closing.

Karthik Raghu Iyer, Yazdan Jamshidi, Nicholas Bray +1

Palo Alto Networks

LLM Safety BenchmarksBenchmark AuditingLLM AuditingAdversarial Attacks on LLMsAI Agent Security BenchmarksLLM Security+1
AskPDF