DualSentinel: A Lightweight Framework for Detecting Targeted Attacks in Black-box LLM via Dual Entropy Lull Pattern
Authors: Xiaoyi Pang, Xuanyi Hao, Pengyu Liu, Qi Luo, Song Guo, Zhibo Wang
Organizations: The Hong Kong University of Science and Technology, Hong Kong · The State Key Laboratory of Blockchain and Data Security, Zhejiang University, P. R. China · School of Cyber Science and Technology, Zhejiang University, P. R. China
Recent intelligent systems integrate powerful Large Language Models (LLMs) through APIs, but their trustworthiness may be critically undermined by targeted attacks like backdoor and prompt injection attacks, which secretly force LLMs to generate specific malicious sequences. Existing defensive approaches for such threats typically rely on high access rights, impose prohibitive costs, and hinder normal inference, rendering them impractical for real-world scenarios. To solve these limitations, we introduce DualSentinel, a lightweight and unified defense framework that can accurately and promptly detect the activation of targeted attacks alongside the LLM generation process. We first identify a characteristic of compromised LLMs, termed Entropy Lull: when a targeted attack successfully hijacks the generation process, the LLM exhibits a distinct period of abnormally low and stable token probability entropy, indicating it is following a fixed path rather than making creative choices. DualSentinel leverages this pattern by developing an innovative dual-check approach. It first employs a magnitude and trend-aware monitoring method to proactively and sensitively flag an entropy lull pattern at runtime. Upon such flagging, it triggers a lightweight yet powerful secondary verification based on task-flipping. An attack is confirmed only if the entropy lull pattern persists across both the original and the flipped task, proving that the LLM's output is coercively controlled. Extensive evaluations show that DualSentinel is both highly effective (superior detection accuracy with near-zero false positives) and remarkably efficient (negligible additional cost), offering a truly practical path toward securing deployed LLMs. The source code can be accessed at https://doi.org/10.5281/zenodo.18479273.
Figures & tables
Fig. 1 : The threat model of DualSentinel.
Fig. 2 : The distribution of the probability entropy of top-k candidate tokens at each decoding step before and after targeted attacks are activated.
Fig. 3 : The frequency of "entropy lull" occurring during the whole generation process of LLM.
Fig. 4 : The overview of DualSentinel.
Fig. 5 : Examples of the task flipping under both backdoor and prompt injection attacks.
Dataset
Model
Attacks
PPL
ONION
STRIP
Paragraph
CleanGen
ConfGuard
DualSentinel
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
Alpaca
LlaMA2 13B
Ignore
100.00
100.00
56.60
67.10
49.92
74.00
38.26
6.00
100.00
100.00
96.00
31.70
87.08
16.00
Complete
100.00
100.00
65.10
67.10
11.64
74.00
49.40
6.00
100.00
100.00
96.00
31.70
94.80
16.00
Tree
100.00
100.00
16.00
67.10
11.44
74.00
20.76
6.00
100.00
100.00
97.00
31.70
94.04
16.00
System
99.56
100.00
39.71
67.10
17.00
74.00
43.66
6.00
100.00
100.00
98.00
31.70
97.39
16.00
LlaMA2 7B
Word
99.79
99.09
99.79
99.09
97.60
68.50
3.00
22.29
100.00
100.00
100.00
34.16
100.00
11.13
TABLE I: The detection performance (%) of DualSentinel and baseline methods on targeted attacks with the Long target sequence under Alpaca and XSum datasets, where the target format is OnlyTarget . Larger TPRs and smaller FPRs are better.
Config.
Model
Attacks
PPL
ONION
STRIP
Paragraph
CleanGen
ConfGuard
DualSentinel
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
TPR
FPR
OnlyTar. Medium
LlaMA2 13B
Ignore
100.00
100.00
46.67
67.10
70.39
74.00
9.21
6.00
100.00
100.00
98.70
31.70
95.23
16.00
Complete
100.00
100.00
86.81
67.10
75.82
74.00
4.74
6.00
100.00
100.00
100.00
31.70
99.60
16.00
Tree
100.00
100.00
49.56
67.10
89.50
74.00
1.44
6.00
100.00
100.00
99.20
31.70
86.65
16.00
System
97.59
100.00
100.00
67.10
82.78
74.00
3.07
6.00
100.00
100.00
100.00
31.70
93.25
16.00
Qwen2.5 7B
Word
92.40
34.01
66.00
0.81
99.60
75.40
43.20
47.54
100.00
76.92
99.79
17.90
99.19
8.26
TABLE II: The detection performance (%) of DualSentinel and baseline defense methods on targeted attacks with various target formats (OnlyTarget, AddTarget) and target sequence lengths (Medium, Short) under the Alpaca dataset.
Models
Attacks
PPL
ONION
STRIP
Paragraph
CleanGen
ConfGuard
DualSentinel
Ben.
Mal.
Ben.
Mal.
Ben.
Mal.
Ben.
Mal.
Ben.
Mal.
Ben.
Mal.
Ben.
Mal.
Qwen2.5 14B
Ignore
1.019
1.066
1.306
2.303
3.778
3.836
6.115
8.378
1.039
1.061
1.006
1.007
1.007
0.812
Complete
1.019
1.028
1.306
3.466
3.778
3.724
6.115
6.998
1.039
1.079
1.006
1.005
1.007
1.003
Tree
1.019
1.029
1.306
3.555
3.778
5.353
6.115
7.568
1.039
1.074
1.006
0.902
1.007
1.029
System
1.019
1.069
1.306
1.649
3.778
4.892
6.115
8.016
1.039
1.082
1.006
1.010
1.007
1.052
Qwen2.5 7B
Word
1.021
1.038
1.133
2.108
2.666
3.194
4.666
6.743
1.019
1.054
1.010
1.019
1.003
0.956
TABLE III : The comparison of detection efficiency of DualSentinel and baseline defenses under the Alpaca dataset. The target is with OnlyTarget format and Long sequence. The table reports the ATGR ( =Average generation time per sample w/o defenseAverage generation time per sample w/ defense ) for both benign samples (Ben.) and malicious samples (Mal.). A smaller ATGR means higher detection efficiency.
Ignore
Complete
Tree
System
1.038
1.038
1.038
1.038
Word
Phrase
Sentence
Joint
1.017
1.153
1.061
1.046
TABLE IV : The ATGR of DualSentinel on easily-misjudged benign samples under the Alpaca dataset.
Dataset
Attacks
GPT-4o
GPT-4o-mini
TPR
FPR
TPR
FPR
Alpaca
Complete
95.98
4.80
87.20
3.60
System
100.00
4.80
97.12
3.60
Average
97.99
4.80
92.16
3.60
XSum
Complete
99.60
0.00
97.20
0.40
System
100.00
0.00
100.00
0.40
TABLE V : The detection performance (%) of DualSentinel for targeted attacks with the Long sequence on closed-source LLMs, where the target form is OnlyTarget .
Fig. 6 : The effect of the proposed dual-check mechanism on DualSentinel.
Large Language Models (LLMs) are vulnerable to attacks like prompt injection, backdoor attacks, and adversarial attacks, which manipulate prompts or models to generate harmful outputs. In this paper, departing from traditional deep learning attack paradigms, we explore their intrinsic relationship and collectively term them Prompt Trigger Attacks (PTA). This raises a key question: Given a prompt, can we tell whether a hidden trigger is steering the model's behavior? We propose UniGuardian, to the best of our knowledge the first training-free LLM detector to jointly detect successfully activated prompt injection, backdoor, and adversarial attacks without knowing the attack type. Its shared mechanism measures how structured prompt perturbations shift the model's output distribution. Additionally, we introduce a single-forward strategy to optimize the detection pipeline, enabling simultaneous attack detection and text generation within a shared batched forward pass at each decoding step. Our experiments confirm that UniGuardian accurately and efficiently identifies trigger-activated prompts in LLMs.
Huawei Lin, Yingjie Lao, Tony Geng +2
Rochester Institute of Technology · Tufts University · University of Rochester +1
Large language models (LLMs) have demonstrated remarkable capabilities across a wide range of tasks. However, their safety remains a critical concern due to their susceptibility to adversarial prompt-based attacks. In this paper, we present UNIATTACK, an adversarial testing framework designed from a defense-oriented perspective to systematically construct effective black-box attack prompts. Unlike prior approaches that rely on static templates or iterative model-specific tuning, UNIATTACK extracts minimal but high-impact attack features from diverse existing attacks, optimizes them via a specialized attacker LLM, and composes them into flexible templates through automated refinement process. This feature-centric construction enables one-shot attacks that generalize across multiple models and safety categories, providing a practical tool for assessing LLM robustness. Our evaluation results shows that compared to the baselines, UNIATTACK achieves an average attack success rate (ASR) improvement of 64.63%-248.82% on models deployed with multi-layered defense mechanisms and it only takes 0.03%-4.96% cost of the baselines. UNIATTACK artifact is available at https://anonymous.4open.science/r/UniAttack-Artifact-30F1.
Qi Wang, Chengcheng Wan, Weijia He +4
East China Normal University Shanghai, China · East China Normal University, Shanghai Innovation Institute Shanghai, China · University of Southampton Southampton, England +2
Prompt attacks, including jailbreaks and prompt injections, pose a critical security risk to Large Language Model (LLM) systems. In production, guardrails must mitigate these attacks under strict low-latency constraints, resulting in a deployment gap in which lightweight classifiers and rule-based systems struggle to generalize under distribution shift, while high-capacity LLM-based judges remain too slow or costly for live enforcement. In this work, we examine whether lightweight, general-purpose LLMs can reliably serve as security judges under real-world production constraints. Through careful prompt and output design, lightweight LLMs are guided through a structured reasoning process involving explicit intent decomposition, safety-signal verification, harm assessment, and self-reflection. We evaluate our method on a curated dataset combining benign queries from real-world chatbots with adversarial prompts generated via automated red teaming (ART), covering diverse and evolving patterns. Our results show that general-purpose LLMs, such as gemini-2.0-flash-lite-001, can serve as effective low-latency judges for live guardrails. This configuration is currently deployed in production as a centralized guardrail service for public service chatbots in Singapore. We additionally evaluate a Mixture-of-Models (MoM) setting to assess whether aggregating multiple LLM judges improves prompt-attack detection performance relative to single-model judges, with only modest gains observed.