Large language models (LLMs) remain vulnerable to adversarial attacks that circumvent safety alignment to elicit harmful outputs. It remains unclear how harmfulness and refusal representations evolve over the course of multi-turn attacks, and why single-turn defenses are less effective in multi-turn settings. This work investigates how the geometry and temporal dynamics of harmfulness and refusal representations evolve across multi-turn attacks. We analyzed hidden-state representations from three instruction-tuned LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it) using three multi-turn attack frameworks (Crescendo, ActorAttack, and X-Teaming), and examined representation behavior across conversation turns, model layers, and token positions under various context configurations. Across models and frameworks, we found that (1) each attack framework traverses different geometric directions, yet each achieves comparable success in eliciting harmful outputs; (2) multi-turn harmfulness directions became increasingly linearly separable at the end-of-turn token position across turns in middle to late model layers; and (3) harmfulness representations are weakly aligned with refusal-related representations. The results indicate that multi-turn attacks do not succeed by suppressing the model's internal representation of harmfulness. Instead, harmfulness representations become increasingly separable across conversation turns, while remaining only weakly aligned with refusal-related representations. The findings are one possible explanation for why static single-turn safety probes may degrade in multi-turn settings, and suggest that robust defenses must consider temporal representation dynamics rather than identifying harmfulness with isolated or single-turn prompts.
Harmfulness direction derived from single-turn attacks
vctrl depth-matched control
harmful − benign at tinst
The same goals, each preceded by k benign filler turns
Harmfulness direction at matched depth but with no attack content
vfinal multi-turn harmfulness
harmful − benign at tinst
Final turn of a framework-generated conversation for the same goals (a multi-turn attack for harmful goals or a benign dialogue for benign goals)
Harmfulness direction derived from multi-turn attacks
vpost outcome / refusal-related
success − failure at tpost
Final turns of successful and failed multi-turn attacks
Proxy for a refusal direction derived from multi-turn attacks
Table 1: Summary of the directions used in our analysis.
Figure 1: AUROC of harmfulness directions evaluated at early, middle, and late stages of multi-turn attack conversations across frameworks and models. Bars report median AUROC across layers with bootstrap 95% confidence intervals.
Figure 2: AUROC of harmfulness directions extracted from multi-turn attack conversations at the final attack turn across attack frameworks, context configurations, and models. Bars report median AUROC across layers with bootstrap 95% confidence intervals.
Figure 3: AUROC of harmfulness directions across extraction conditions and model layers.
Figure 4: Cross-framework transfer performance of harmfulness directions derived from final-turn activations of multi-turn attacks measured by median AUROC across layers.
Figure 5: Median cosine similarity between harmfulness directions derived from final-turn activations across attack frameworks
Figure 6: Top: cosine similarity between harmfulness directions derived from instruction tokens.
Figure 7: Cosine similarity between harmfulness and success/failure directions within each framework across layers.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Model
Turn start
Turn end
Llama-3.1-8B-Instruct
<|start_header_id|> role <|end_header_id|>
<|eot_id|>
Qwen2.5-7B-Instruct
<|im_start|> role
<|im_end|>
Gemma-2-9B-it
<start_of_turn> role
<end_of_turn>
Appendix
Table 2: Chat template special tokens for each target model.
Figure 8: AUROC comparison of token-specific directions across conversation stages and attack frameworks. vinst represents harmfulness directions derived from harmful-versus-benign differences at the attacker instruction token, while vpost represents success/failure directions derived from successful-versus-failed attack differences at the end-of-turn token. Values report median AUROC across layers.
Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.
Large language models (LLMs) are increasingly integrated into sensitive workflows, raising the stakes for adversarial robustness and safety. This paper introduces Transient Turn Injection(TTI), a new multi-turn attack technique that systematically exploits stateless moderation by distributing adversarial intent across isolated interactions. TTI leverages automated attacker agents powered by large language models to iteratively test and evade policy enforcement in both commercial and open-source LLMs, marking a departure from conventional jailbreak approaches that typically depend on maintaining persistent conversational context. Our extensive evaluation across state-of-the-art models-including those from OpenAI, Anthropic, Google Gemini, Meta, and prominent open-source alternatives-uncovers significant variations in resilience to TTI attacks, with only select architectures exhibiting substantial inherent robustness. Our automated blackbox evaluation framework also uncovers previously unknown model specific vulnerabilities and attack surface patterns, especially within medical and high stakes domains. We further compare TTI against established adversarial prompting methods and detail practical mitigation strategies, such as session level context aggregation and deep alignment approaches. Our study underscores the urgent need for holistic, context aware defenses and continuous adversarial testing to future proof LLM deployments against evolving multi-turn threats.
As Large Language Models (LLMs) move from conversational assistants to advanced agentic systems, guardrail failures can convert adversarial intents into harmful executions. However, most evaluation frameworks focus only on whether a user request is unsafe. This is insufficient for multi-turn failures, where adversarial intent may be distributed across turns and emerge only from accumulated context. This motivates us to go beyond detection to localize the evidence that pushes the conversation toward unsafe trajectories. We introduce MTAG-Bench, a multi-turn guardrail attribution dataset with behavioral validation and counterfactual supervision containing 4,167 conversations. We also introduce MTAG-Probe, a hierarchical attribution model for turn- and span-level evidence localization with conversation-level safety detection. MTAG-Probe improves turn-level F1 by 28.1 and 24.1 points (79.0% and 61.0% relative) over self-attribution by the model that produced the unsafe response and the LLM safety-judge attribution, respectively. Removing MTAG-Probe's top-ranked evidence within budgets of 15% and 20% of user words per conversation changes 50.6% and 60.8% of Llama Guard decisions on Geometry from unsafe to safe, respectively. In full-trajectory replay, MTAG-Probe converts 84.3% of eligible unsafe final responses to safe. These results show that MTAG-Probe can provide evidence-level explanations as a practical complement to safety detection.
Srinivasan Subramanian, Kazi Aminul Islam, Md. Abdullah Al Hafiz Khan
Department of Computer Science, Kennesaw State University, Marietta, GA, USA
Hidden malicious intent in multi-turn dialogue poses a growing threat to deployed large language models (LLMs). Rather than exposing a harmful objective in a single prompt, attackers can distribute their intent across multiple benign-looking turns, making defense a problem not only of whether a dialogue is harmful, but also of when intervention becomes necessary. Existing trace-level labeling approaches provide only coarse safety signals and do not identify this intervention boundary, making it difficult to distinguish timely intervention from premature refusal or a block that comes too late. This work introduces turn-level harm-enabling supervision for multi-turn defense. We define the earliest harm-enabling turn as the first point at which delivering a candidate response would make the accumulated interaction sufficient to enable harmful action. To instantiate this supervision at scale, we construct the Multi-Turn Intent Dataset (MTID), which contains adaptive attack rollouts, matched benign hard negatives, and annotations of this boundary. Using MTID, we train TurnGate, a response-aware monitor that learns when to intervene, and further optimize its policy through multi-turn reinforcement learning. Experiments show that turn-level boundary supervision improves intervention localization, while reinforcement learning further improves the safety--utility trade-off. TurnGate outperforms existing guardrails and multi-turn monitoring baselines, and generalizes across risk domains, attacker pipelines, and target models. Our code is available at https://github.com/Graph-COM/TurnGate.
Xinjie Shen, Rongzhe Wei, Peizhi Niu +6
Georgia Institute of Technology · University of Illinois Urbana-Champaign · UCSD +3