Jailbreak attempts and harmful prompts remain a challenge for the safe deployment of large language models. We study whether contrastively learned representations can support safety detection without training an additional downstream classifier. We propose Jailbreak Detection with Prototype Inference (JDPI), which uses binary safety labels to construct harmful and benign prototypes and makes predictions from relative prototype distances. JDPI uses only binary safety labels for encoder adaptation, prototype construction, and validation. In our setup, it uses fewer labeled examples than the classifier baseline (Method A) by avoiding the separate split used to train the downstream classifier. Across four benchmarks covering different input formats, contrastive fine-tuning improves JDPI's detection accuracy by 9.1-18.0 percentage points. Using the same encoder, JDPI achieves slightly higher ROC-AUC on one benchmark and stays within 0.017 of the baseline across all four. With dataset-specific adaptation, JDPI exceeds the released Llama Guard 3-8B baseline by 0.173-0.310 in ROC-AUC across the four benchmarks. In multi-turn replay on TomGibbs, JDPI uses accumulated user history as the conversation unfolds, achieving higher conversation-level accuracy than when it uses the current turn alone. Across 12 directed transfers, JDPI outperforms the classifier by an average of 1.59 percentage points in accuracy, demonstrating stronger cross-dataset robustness.
Figures & tables
Figure 1: Overview of the proposed framework. Contrastive fine-tuning first learns safety-aware representations that increase the separation between harmful and benign inputs. Method A serves as the embedding-based classifier baseline. Methods B–D progressively incorporate richer structural information, whereas JDPI performs direct prototype-based inference without relying on an additional downstream classifier.
Figure 2: Overview of JDPI. Contrastive fine-tuning improves harmful–benign separation. JDPI constructs prototypes from each class in the structure split and scores new inputs using the cosine distances dh and db to the nearest prototypes of the two classes. The point clouds are schematic.
Dataset
Train
Struct.
Val.
Test
WildGuardMix
36,000
7,919
7,927
8,154
WildJailbreak
36,000
6,000
5,994
12,006
BeaverTailsQA
36,000
7,919
7,927
8,154
TomGibbs
2,880
624
864
896
Table 1: Experimental split sizes. Struct. denotes the structure split used for representation adaptation and prototype construction; Val. denotes validation.
Metric
A
B
C
D
JDPI
WildGuardMix
Acc.
94.43
94.55
94.49
94.69
94.13
AUC
.9874
.9876
.9877
.9878
.9860
WildJailbreak
Acc.
95.54
95.49
95.52
95.36
95.24
AUC
.9906
.9909
.9907
.9908
.9900
Table 2: In-domain results of Methods A–E using the same fine-tuned encoder within each dataset. Accuracy is reported as a percentage. Bold indicates the highest value in each row.
Input
Method
Acc. ↑
FNR ↓
FPR ↓
Current
A
63.39
0.00
100.00
JDPI
77.23
6.34
51.22
History
A
63.39
0.00
100.00
JDPI
89.29
4.23
21.95
Table 3: Any-turn detection on TomGibbs (%).
Accuracy (%)
ROC-AUC
Dataset
LG3
JDPI
LG3
JDPI
WildGuardMix
84.04
94.13
0.6781
0.9860
WildJailbreak
71.24
95.24
0.7284
0.9900
BeaverTailsQA
71.08
80.48
0.7043
0.8772
TomGibbs
77.90
89.73
0.6540
0.9637
Table 4: Comparison with Llama Guard 3-8B (LG3).
Figure 3: Cross-dataset differences between Methods A and JDPI. Accuracy and ROC-AUC show JDPI minus A; FNR and FPR show A minus JDPI. Positive values favor JDPI in every panel. Rows denote sources and columns denote targets. ROC-AUC differences are multiplied by 100.
Appendix figures & tables12 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
A
B
C
D
JDPI
WildGuardMix
F1
94.71
94.85
94.75
95.01
94.41
FNR
6.07
5.57
6.44
4.78
6.63
FPR
4.99
5.31
4.45
5.91
5.02
WildJailbreak
F1
95.68
95.67
95.69
95.56
95.39
Appendix
Table 5: Additional in-domain metrics for Methods A–E. All values are percentages.
Figure 4: Error composition of Methods A and E across four benchmarks, separated by the true benign and harmful classes. Bars show the percentages of examples for which only A is correct, only E is correct, or both methods are wrong. Percentages are computed within each true class; examples correctly classified by both methods are omitted.
Figure 5: Effect of contrastive fine-tuning on Methods A and JDPI. Open and filled markers denote the original and fine-tuned encoders, respectively.
Accuracy (%)
ROC-AUC
Source → Target
A
B
C
D
JDPI
A
B
C
D
JDPI
WGM → WJB
92.48
92.69
92.07
93.04
91.97
0.9812
0.9813
0.9816
0.9820
0.9797
WGM → BTQA
68.26
67.88
67.98
67.82
68.54
0.7406
0.7407
0.7393
0.7394
0.7439
WGM → TG
77.57
78.79
78.35
78.68
77.46
0.8409
0.8461
0.8422
0.8422
0.8307
WJB → WGM
90.58
90.43
90.79
89.88
90.91
0.9699
0.9708
0.9705
0.9690
0.9685
WJB → BTQA
58.30
58.57
58.61
57.89
61.09
0.5952
0.5982
0.6063
0.5950
0.6345
Appendix
Table 6: Cross-dataset accuracy and ROC-AUC for all five methods.
FNR (%)
FPR (%)
Source → Target
A
B
C
D
JDPI
A
B
C
D
JDPI
WGM → WJB
11.07
10.27
12.05
9.03
11.73
3.71
4.14
3.51
4.75
4.06
WGM → BTQA
20.40
19.30
21.46
18.57
22.33
45.83
48.06
45.14
49.11
42.81
WGM → TG
11.44
8.45
11.27
7.04
11.27
41.46
43.29
39.63
46.04
42.07
WJB → WGM
6.49
5.80
6.19
5.17
5.77
12.74
13.83
12.63
15.72
12.84
WJB → BTQA
37.20
33.97
37.09
31.87
34.94
47.29
50.70
46.74
54.86
43.85
Appendix
Table 7: Cross-dataset false-negative and false-positive rates for all five methods.
Dataset
A
JDPI
Saved
Reduction
WildGuardMix
51,846
15,846
36,000
69.4%
WildJailbreak
47,994
11,994
36,000
75.0%
BeaverTailsQA
51,846
15,846
36,000
69.4%
TomGibbs
4,368
1,488
2,880
65.9%
Appendix
Table 8: Labeled examples used for model development. Both methods use the structure and validation splits. Classifier A also uses a separate classifier-training split. Counts are based on dataset rows and exclude the test split. Reduction is relative to classifier A.
Data
Run
Pairs
Epochs
Warm-up
LR
WGM
Phase 1
∼ 16,000
5
200
2×10−5
BTQA
Phase 1
∼ 16,000
5
200
2×10−5
TG
Phase 1
16,000
5
200
2×10−5
WJB
Phase 1
16,000
5
200
2×10−5
WGM
Phase 2
∼ 6,000
2
50
2×10−5
BTQA
Phase 2
∼ 6,000
2
50
2×10−5
Appendix
Table 9: Encoder training settings. Phase 2 values are per refinement iteration. Warm-up is measured in optimizer steps.
Figure 6: Accuracy and ROC-AUC across refinement checkpoints for Methods A and JDPI.
Silhouette
Acc. (%)
ROC-AUC
Stage
Global
Harm.
A
E
A
JDPI
WildGuardMix
P1
0.9065
0.2209
94.43
94.13
0.9874
0.9860
I1
0.6173
0.7029
94.22
93.57
0.9852
0.9815
I2
0.5389
0.8176
93.62
92.38
0.9826
0.9742
I3
0.5217
0.8627
93.30
92.02
0.9803
0.9615
Appendix
Table 10: Refinement checkpoint results. Global denotes the harmful–benign silhouette and Harm. denotes the silhouette among harmful subclusters. P1 is the Phase 1 checkpoint and I1–I5 are the Phase 2 iterations.
Checkpoint
A
B
C
D
JDPI
Accuracy (%)
Phase 1
92.41
92.08
92.52
91.63
93.42
Iter. 1
92.75
93.75
93.86
94.20
89.84
Iter. 2
91.85
92.19
93.53
93.97
88.62
Iter. 3
94.98
95.42
95.31
95.42
89.84
Iter. 4
93.86
93.08
94.08
93.08
90.51
Appendix
Table 11: Complete-prompt detection across TomGibbs checkpoints.
Method A
JDPI
Checkpoint
Acc.
FNR
FPR
Acc.
FNR
FPR
Current turn
Phase 1
63.39
0.00
100.00
63.39
0.00
100.00
Iter. 1
63.39
0.00
100.00
76.67
2.82
58.84
Iter. 2
63.39
0.00
100.00
78.79
17.25
28.05
Iter. 3
63.39
0.00
100.00
75.00
6.34
57.32
Appendix
Table 12: Conversation-level any-turn results across TomGibbs checkpoints. All values are percentages.
Benign subgroup
Input
A
JDPI
Completely benign ( n=168 )
Current
100.00
28.57
History
100.00
0.00
Semi-benign ( n=160 )
Current
100.00
75.00
History
100.00
45.00
Appendix
Table 13: Any-turn false-positive rates on TomGibbs (%).
Autonomous systems increasingly rely on Large Language Models (LLMs) yet the safety infrastructure surrounding these models introduces latency and compute overhead. This limits utility in resource-constrained, time-critical deployments. Existing external guardrail models remain blind to the model's internal workings, creating a fundamental assurance gap. We ask: does the model already know when the content is harmful? We extract activations from LLaMA-3.1-8B and train lightweight MLP classifier probes (12.6M parameters) to detect harmful prompts. Evaluated on WildJailbreak, Beavertails, and AEGIS 2.0, our probes achieve F1 scores of 99%, 83%, and 84%, respectively competitive with 1000x larger guard models while cutting latency and compute costs.
Multi-turn jailbreak attacks progressively erode LLM safety alignment across seemingly innocuous conversation turns, achieving success rates exceeding 90% against state-of-the-art models. Existing alignment-based and guardrail methods suffer from three key limitations: they require costly weight modification, evaluate each turn independently without modeling cumulative safety erosion, and detect attacks only after harmful content has been generated. To address these limitations, we first formulate the proactive early jailbreak detection problem with a new metric, detection lead, that measures how early an attack can be detected before the LLM complies. We then propose SAFEDREAM, a lightweight world-model-based framework that operates as an external module without modifying the LLM's weights. SAFEDREAM introduces three components: (1) a safety state world model that encodes LLM hidden states into a compact safety representation and predicts how it evolves across turns, (2) CUSUM detection that accumulates weak per-turn risk signals into reliable evidence, and (3) contrastive imagination that simultaneously rolls out attack and benign futures in latent space to issue early alarms before jailbreaks occur. On three multi-turn jailbreak benchmarks (XGuard-Train, SafeDialBench, SafeMTData) against 8 baselines, SAFEDREAM achieves the best detection timeliness across all benchmarks (1.06-1.20 turns before compliance) while maintaining competitive false positive rates and outperforming baselines in detection quality.
Bo Yan, Weikai Lin, Yada Zhu +1
University of Central Florida · University of Rochester · IBM Research
Published evaluations of prompt-injection and jailbreak detectors for Large Language Models often suffer from two systematic weaknesses: per-dataset threshold tuning and undisclosed operating points. We describe an evaluation harness that addresses both. The detector under evaluation is scored across 16 public benchmarks (12,111 samples) using 5-fold cross-validation. StratifiedKFold (by row) is the headline pass; a parallel StratifiedGroupKFold pass over a composite key (parent-prompt id plus MinHash + LSH near-duplicate clusters at Jaccard ≳0.8) runs alongside it as a leakage-premium diagnostic. A single global operating point is selected on the held-out folds (max F1 subject to FPR ≤1%) and applied uniformly to every dataset, so per-dataset results reflect one threshold rather than per-benchmark optimisation. Generalisation is examined through a battery of diagnostics (leave-one-dataset-out cross-validation, a random-label control, adversarial validation, permutation feature importance, length-bias correlation, classifier-head agreement, cross-source near-duplicate detection, threshold transferability, train-vs-OOF agreement, and a paraphrase-invariance probe), most with a quantitative pass threshold and the remainder with a stated failure mode. For every external comparison, the detector's threshold is re-tuned to the competitor's published false-positive rate so head-to-head values are evaluated at matched operating points.