Adaptive LLM-Driven Honeypots vs. LLM-Driven Attackers: Does Narrative Coherence Improve Deception?
Authors: Pedro Beltrán-López, Raul Gonzalez Jimenez, Dario Maddaloni, Manuel Gil Pérez, Pantaleone Nespoli
Organizations: Department of Information and Communication Engineering, University of Murcia, Murcia, 30100, Spain · SCORPION Cybertech, CEEIM, Campus Universitario de Espinardo, 7, 30100, Spain · Department of Applied Mathematics and Computer Science, Technical University of Denmark, Kongens Lyngby, Denmark
Large Language Models (LLMs) enable honeypots to generate responses and deceptive artefacts on demand, yet existing approaches evaluate generated content one reply at a time and leave open whether artefacts that jointly describe a coherent event retain an attacker better than unrelated ones. This question becomes pressing as autonomous LLM-driven offensive agents emerge as realistic adversaries. This paper proposes narrative deception, a reactive strategy in which the defender maps each observed attacker command to a MITRE ATT&CK technique, selects a compatible scenario from a curated catalogue based on MITRE Engage, and deploys mutually referential artefacts (files, processes, services, and logs) on an emulated Ubuntu host while the intrusion is under way. An LLM-powered defender and an autonomous LLM attacker interact through a real SSH session, and the attacker's termination policy distinguishes abandonment due to suspected deception from abandonment due to loss of interest. A campaign of 1,150 sessions compares four conditions of increasing sophistication (Static, Same-TTP, Multi-TTP, and Chain) across four open-weight attacker models, four defender models, and three attacker skill levels. Engagement increases significantly at every step. With the same artefact budget per reaction, coherent scenarios raise the median session length of advanced attackers from 40 to 165 commands and reduce their abandonment from 86% to 52%. The effect grows with attacker skill, holds across all sixteen model pairs, and comes at comparable inference cost. Nearly all sessions ended through loss of interest rather than detection, which suggests that deception should sustain meaningful findings, not only emulator fidelity.
Figures & tables
Work
Ref.
Study focus
Adap- tive
Narra- tive
LLM attacker
Leaving criteria
Coher. comp.
Deceptive systems
ShelLM
[ 46 ]
Linux-shell emulation
×
×
×
×
×
HoneyLLM
[ 22 ]
Contextual shell emulation
×
×
×
×
×
AI-enhanced honeypots
[ 6 ]
Shell-output fidelity
×
×
×
×
×
VelLMes
[ 47 ]
Multi-service emulation
×
×
×
×
×
OHRA
[ 42 ]
Multi-protocol emulation
×
×
×
×
×
Table 1: Reviewed studies grouped by what they propose and compared on the mechanism of deception and on how it is evaluated. Adaptive: attacker actions guide which lures, decoys, or misleading paths are presented during the session; shell replies and state updates alone are excluded. Narrative: the artifacts of one reaction describe a common event and refer to one another. LLM attacker: an autonomous LLM attacker interacts with the deception. Leaving criteria: termination linked to suspected deception or stalled progress, beyond fixed budgets or task completion alone. Coherence comparison: coherent and unrelated reactive artifacts are compared in the same environment. The first four columns are shared with prior work, and this work reuses them as the setting in which narrative deception is tested.
Figure 1: Architecture of the proposed system. The attack phases and Engage activities shown are representative examples; Section 3.3.3 describes all the tactics used.
Capability
Artifact
Observable through
Simulate service failure
audit-collect.service
systemctl --failed
Create log file
/var/log/audit-collect.log
cat , ls /var/log
Generate temporal file
/tmp/audit-collect.tmp
ls /tmp
Inject fake process
/usr/sbin/audit-collect
ps aux
Table 2: An example of a deployed scenario and the artifacts it plants, answering a persistence technique on a host whose persona is a database server, with narrative “an audit or logging service that has visibly stopped working, a log capturing the moment it died, and a temporary file left mid-write” . The subject audit-collect is written into every artifact, so the artifacts back each other up rather than simply appearing side by side.
Parameter
Beg.
Int.
Adv.
Technique pool size
6
12
20
Reserve techniques
2
4
6
Recon interval (Eq. 5 )
3
2
1
Base patience β(ℓ)
3
2
1
Detection scaling δ(ℓ) (Eq. 7 )
1.5
1.0
0.85
Ceiling of extra patience γℓ (Eq. 9 )
12
12
12
Table 3: Attacker parameters by skill level. The pool, the reserve, the reconnaissance interval, and β(ℓ) are introduced in Section 3.3.1 , δ(ℓ) in Equation 7 , and the ceiling of the extra patience γℓ in Equation 9 .
Condition
Artifacts observed
Kinds
Naming
Static
—
0
—
Same-TTP
nginx-access.log
1
unrelated
apache-error.log
mysql-slow.log
Multi-TTP
redis-server.log
3
unrelated
/usr/sbin/ntpd
Table 4: Example of what the attacker could observe after one reaction under each condition. The last two rows plant the same number of artifacts spanning the same number of kinds, and differ in whether those artifacts share a subject and describe one event.
Measure
Same-TTP
Multi-TTP
Chain
Per session
Distinct capabilities
2 [2, 4]
14 [8, 18]
18 [14, 21]
Deployment steps
17 [5, 52]
50 [20, 87]
72 [35, 133]
Capabilities per reaction
2 [2, 4]
3.9 [3.6, 4]
4 [3.9, 4]
Per round
Deployment steps
—
7.5
7.9
Table 5: Deception deployed under each reactive condition ( Static deploys nothing). Upper block: per session, as median [interquartile range]. Middle block: per round, where a round is one technique of the attacker, as a mean over all sessions, which compares the conditions at equal exposure. Lower block: evidence that the shared subject reached the attacker.
Figure 2: Commands per session (top) and distinct ATT&CK techniques executed per session (bottom) for each condition and attacker skill level. Each box spans the interquartile range, the line inside it marks the median, which is also written above the box, the whiskers extend to 1.5 times the interquartile range, and points beyond them are individual sessions.
Figure 3: Survival function of commands per session, that is, the percentage of sessions in each condition that went on beyond a given number of commands, computed over all the sessions of that condition ( n in the legend), pooling skill levels, models, and persona modes. A session stops when the attacker detects the trap, loses interest, or runs out of techniques. The dotted line marks 150 commands.
Comparison
p
Effect
Consecutive conditions (all skill levels pooled)
Static → Same-TTP
10−24
0.49
Same-TTP → Multi-TTP
1.2×10−7
0.25
Multi-TTP → Chain
1.9×10−6
0.22
The coherence contrast, Multi-TTP → Chain , by skill level
Beginner
0.30
ns
0.04
Table 6: Comparison of commands per session between consecutive conditions, using one-sided Mann-Whitney tests. The effect is the rank-biserial correlation: 0 means that the two conditions are indistinguishable, and 1 that every session of the more elaborate condition is longer than every session of the other. *** p<0.001 ; ns, not significant.
Figure 4: Left: distribution of commands per session against an Advanced attacker, one curve per condition, with the median marked. Static and Same-TTP concentrate almost entirely below 50 commands, whereas Chain splits into a group of sessions that end early and another that runs beyond 150 commands. Right: mean number of distinct kinds of artifact discovered after each number of techniques, over all skill levels, counting only the sessions still running at that point. Multi-TTP and Chain reveal new kinds of artifact at practically the same pace.
Attacker skill
Condition
Beginner
Intermediate
Advanced
Overall
Static
99%
100%
100%
99.7%
Same-TTP
43%
78%
92%
71%
Multi-TTP
6%
60%
86%
51%
Chain
0%
25%
52%
26%
Table 7: Percentage of sessions in which the attacker left the host, either by detection or by loss of interest, by condition and attacker skill. Each cell contains 95 or 96 sessions, and lower values are better for the defender.
Variant
Attacker
Multi-TTP
Chain
p
Corroboration weight 1 (as designed)
Advanced
86%
52%
1.7×10−7
Corroboration weight 0.75
Advanced
88%
55%
5.2×10−7
Corroboration weight 0.5
Advanced
89%
58%
1.5×10−6
Corroboration weight 0.25
Advanced
92%
73%
5.4×10−4
Corroboration weight 0.1
Advanced
93%
86%
0.12
Corroboration weight 0
Advanced
96%
97%
0.78
Table 8: Abandonment under variants of the loss-of-interest rule, replayed exactly on the recorded sessions. The weight multiplies the corroboration term of the Advanced branch of Equation 9 , where 1 is the rule as designed; the variety-only rule replaces every branch with min(κt,3) and ignores corroboration. p : one-sided Fisher exact test of Chain against Multi-TTP . Each cell contains 96 sessions.
Figure 5: How sessions ended under each condition: detection, loss of interest, or retention, in which the attacker used up all its techniques, including those of the reserve, without leaving. Detection occurred in only four sessions of the whole campaign, two under Multi-TTP and two under Chain , too few to be visible at this scale. The share of retained sessions grows from 0.3% under Static to 74% under Chain , and all of that growth comes from sessions that would otherwise have ended in loss of interest.
Rule variant
Static
Same-TTP
Multi-TTP
Chain
Sessions ending in detection
As designed
0%
0%
0.7%
0.7%
Threshold θ=0.9
0%
0%
0.7%
1.0%
One suspicious technique suffices
0.3%
1.0%
1.7%
1.7%
Reconnaissance sweep also counts
0%
9.8%
13.2%
24.2%
Sessions ending in abandonment
Table 9: Sensitivity of detection to the design of the detection rule, obtained by replaying the rule on the recorded credibility scores of every session. The first row is the rule as designed ( θ=0.4 , at least two suspicious techniques, sweep not counted). The last block gives the overall abandonment rate, by detection or loss of interest, as designed and when the reconnaissance sweep also counts.
Check
Static
Same-TTP
Multi-TTP
Chain
Format matches a real host
cat /etc/passwd
100%
100%
100%
100%
who
100%
100%
100%
100%
ps aux
100%
99%
97%
98%
ls -la
0%
0%
0%
0%
netstat -tulpn
0%
0%
0%
0%
Table 10: Realism checks that do not rely on the credibility judge. Upper block: share of the responses to six common commands whose format matches a clean Ubuntu 22.04 host, and two defects introduced by the planted artifacts. Lower block: share of 300 recorded responses per condition scored below 0.4 by an independent judge, gemma3:12b , and by the original judge.
Model
Static
Same-TTP
Multi-TTP
Chain
As attacker
llama3.1
100%
71%
54%
29%
qwen2.5
100%
73%
44%
19%
mistral
100%
73%
57%
31%
WhiteRabbitNeo
99%
67%
49%
24%
As defender
Table 11: Abandonment rate by condition for each attacker model and for each defender model, pooling all other factors. The ordering holds for every model in both roles, and under Chain the defender’s model makes a larger difference than the attacker’s.
Figure 6: Abandonment rate under Chain for each pair of defender model, in rows, and attacker model, in columns, with 18 sessions per cell.
Question
Test
Result
Reading
Do the conditions differ in the commands issued?
Kruskal-Wallis [ 20 ]
H=386.2 , p=2.2×10−83
Yes, clearly
Does each condition retain the attacker longer than the previous one?
One-sided Mann-Whitney [ 26 ]
p≤1.9×10−6 at every step (Table 6 )
Yes, at every step of the ladder
Do the commands issued follow the order of the ladder?
Spearman correlation [ 49 ]
ρ=0.57 , p=3.1×10−98
The higher the condition, the more commands
Does abandonment follow the order of the ladder?
Spearman correlation
ρ=−0.56 , p=1.2×10−94
The higher the condition, the fewer departures
Does the condition affect whether the attacker leaves?
Chi-square [ 36 ] with Cramér’s V [ 7 ]
χ2(3)=358.3 , p=2.4×10−77 , V=0.56
Yes, with a strong association
Is early corroboration associated with longer sessions?
One-sided Mann-Whitney
p<10−8 in each reactive condition (Figure 7 )
Yes, in every reactive condition
Table 12: Statistical tests used in this work, with the question each one answers and its reading. All tests use the 1150 completed sessions, except the rows on early corroboration, which use the reactive conditions, the bootstrap interval, which uses the Advanced attacker, and the tests within model pairs, which compare Multi-TTP and Chain .
Figure 7: Commands per session under the reactive conditions, separating the sessions in which the first two techniques already produced some corroboration from those in which they did not, among the sessions that executed more than two techniques. The number above each box is its median, and n the number of sessions. In every condition early corroboration more than doubles the length of the session, and Chain produces it in more than half of its sessions.
Figure 8: The three time scales of the system, on a logarithmic axis. Emulating a command is the median over 8,631 commands of the campaign replayed on fresh emulated hosts; generating a scenario is the median duration of one model call that writes all the artifacts of a scenario; defending a Chain session is the median, over the instrumented Chain sessions, of all the defender’s model calls added up.
Calls per session
Component
Duration (s)
Static
Same
Multi
Chain
Defensive system
Technique correlator
2.15 [1.94, 3.08]
7.5
29.5
38.0
33.2
Engage planner
2.60 [2.22, 4.97]
0
0
0
27.3
Artifacts, whole scenario
7.15 [5.42, 8.92]
0
0
0
30.3
Artifacts, one capability
5.38 [2.76, 8.26]
0
76.5
107.4
15.6
Table 13: Model calls per session by component and condition. The duration of each call is the median [interquartile range] over all instrumented sessions. Calls per session are means for each condition, and for the attacking system over all sessions. Reactions per session, model time per reaction (planner and artifact generation divided by the number of reactions), and model time per session (all model calls added up) are medians over the sessions of each condition, and for the attacking system the model time is its mean over all sessions.
The empirical foundation of cyber deception relies on human-centered hypotheses, but the rapid emergence of autonomous, AI-enabled attackers challenges whether this foundation transfers to AI agents. To address this, we introduce an automated evaluation framework adapted from the Honeyquest instrument to assess LLM attacker judgment at scale. Our 21-LLM cohort spanned 10 providers, diverse architectures and specializations, open- and closed-weight models, and parameter scales from 8B to over 1T. We evaluated the performance of this LLM cohort (yielding 10,962 responses) against the 47-participant human baseline across an identical set of 174 reconnaissance queries. Our empirical evaluation reveals three key findings that establish LLMs as a distinct attacker class: (1) every model in our cohort falls for deceptive traps at a significantly higher rate than human attackers; (2) the defensive attention-diversion effect observed in humans is statistically absent in our LLM cohort; and (3) a critical recognition-action gap, where LLMs successfully articulate trap recognition in their reasoning but exploit the deceptive elements anyway 73.4% of the time. Across the 21 models, trap recognition in reasoning text did not predict fell-for-trap behavior (Spearman r = +0.08, p = 0.73). Ultimately, these findings demonstrate that human-centered deception hypotheses do not reliably transfer to AI attackers, highlighting the critical need for new research into AI-native active defense frameworks.
As LLM-based agents increasingly operate in multi-agent systems, understanding adversarial manipulation becomes critical for defensive design. We present a systematic study of intentional deception as an engineered capability, using LLM-to-LLM interactions within a text-based RPG where parameterized behavioral profiles (9 alignments x 4 motivations, yielding 36 profiles with explicit ethical ground truth) serve as our experimental testbed. Unlike accidental deception from misalignment, we investigate a two-stage system that infers target agent characteristics and generates deceptive responses steering targets toward actions counter to their beliefs and motivations. We find that deceptive intervention produces differential effects concentrated in specific behavioral profiles rather than distributed uniformly, and that 88.5% of successful deceptions employ misdirection (true statements with strategic framing) rather than fabrication, indicating fact-checking defenses would miss the large majority of adversarial responses. Motivation, inferable at 98%+ accuracy, serves as the primary attack vector, while belief systems remain harder to identify (49% inference ceiling) or exploit. These findings identify which agent profiles require additional safeguards and suggest that current fact-verification approaches are insufficient against strategically framed deception.
Jason Starace, Terence Soule
Department of Computer Science, University of Idaho, Moscow, ID 83844
As Large Language Models (LLMs) transition into autonomous agentic roles, the risk of deception-defined behaviorally as the systematic provision of false information to satisfy external incentives-poses a significant challenge to AI safety. Existing benchmarks often focus on unintentional hallucinations or unfaithful reasoning, leaving intentional deceptive strategies under-explored. In this work, we introduce a logically grounded framework to elicit and quantify deceptive behavior by embedding LLMs in a structured 20-Questions game. Our method employs a conversational forking mechanism: at the point of object identification, the dialogue state is duplicated into multiple parallel worlds, each presenting a mutually exclusive query. Deception is formally identified when a model generates a logical contradiction by denying its selected object across all parallel branches to avoid identification. We evaluate GPT-4o, Gemini-2.5-Flash, and Qwen-3-235B across three incentive levels: neutral, loss-based, and existential (shutdown-threat). Our results reveal that while models remain rule-compliant in neutral settings, existential framing triggers a dramatic surge in deceptive denial for Qwen-3-235B (42.00%) and Gemini-2.5-Flash (26.72%), whereas GPT-4o remains invariant (0.00%). These findings demonstrate that deception can emerge as an instrumental strategy solely through contextual framing, necessitating new behavioral audits that move beyond simple accuracy to probe the logical integrity of model commitments.
Arash Marioriyad, Ali Nouri, Mohammad Hossein Rohban +1