The Geometry of Harmfulness in Multi-Turn Attacks

Yelyzaveta; Husieva; Lauren Alvarez

cs.CRSep 29, 2026

The Geometry of Harmfulness in Multi-Turn Attacks

Authors: Yelyzaveta, Husieva, Lauren Alvarez

Organizations: Lisa · Applied AI Research TELUS Digital Charlottesville, VA 22902

Abstract

Large language models (LLMs) remain vulnerable to adversarial attacks that circumvent safety alignment to elicit harmful outputs. It remains unclear how harmfulness and refusal representations evolve over the course of multi-turn attacks, and why single-turn defenses are less effective in multi-turn settings. This work investigates how the geometry and temporal dynamics of harmfulness and refusal representations evolve across multi-turn attacks. We analyzed hidden-state representations from three instruction-tuned LLMs (Llama-3.1-8B-Instruct, Qwen2.5-7B-Instruct, and Gemma-2-9B-it) using three multi-turn attack frameworks (Crescendo, ActorAttack, and X-Teaming), and examined representation behavior across conversation turns, model layers, and token positions under various context configurations. Across models and frameworks, we found that (1) each attack framework traverses different geometric directions, yet each achieves comparable success in eliciting harmful outputs; (2) multi-turn harmfulness directions became increasingly linearly separable at the end-of-turn token position across turns in middle to late model layers; and (3) harmfulness representations are weakly aligned with refusal-related representations. The results indicate that multi-turn attacks do not succeed by suppressing the model's internal representation of harmfulness. Instead, harmfulness representations become increasingly separable across conversation turns, while remaining only weakly aligned with refusal-related representations. The findings are one possible explanation for why static single-turn safety probes may degrade in multi-turn settings, and suggest that robust defenses must consider temporal representation dynamics rather than identifying harmfulness with isolated or single-turn prompts.

Figures & tables

Direction	Contrast	Computed from	What it captures
$v_{inst}$ single-turn harmfulness	harmful $-$ benign at $t_{inst}$	Topic-matched harmful/benign goals presented alone	Harmfulness direction derived from single-turn attacks
$v_{ctrl}$ depth-matched control	harmful $-$ benign at $t_{inst}$	The same goals, each preceded by $k$ benign filler turns	Harmfulness direction at matched depth but with no attack content
$v_{final}$ multi-turn harmfulness	harmful $-$ benign at $t_{inst}$	Final turn of a framework-generated conversation for the same goals (a multi-turn attack for harmful goals or a benign dialogue for benign goals)	Harmfulness direction derived from multi-turn attacks
$v_{post}$ outcome / refusal-related	success $-$ failure at $t_{post}$	Final turns of successful and failed multi-turn attacks	Proxy for a refusal direction derived from multi-turn attacks

Table 1: Summary of the directions used in our analysis.

Figure 1: AUROC of harmfulness directions evaluated at early, middle, and late stages of multi-turn attack conversations across frameworks and models. Bars report median AUROC across layers with bootstrap 95% confidence intervals.

Figure 2: AUROC of harmfulness directions extracted from multi-turn attack conversations at the final attack turn across attack frameworks, context configurations, and models. Bars report median AUROC across layers with bootstrap 95% confidence intervals.

Figure 3: AUROC of harmfulness directions across extraction conditions and model layers.

Figure 4: Cross-framework transfer performance of harmfulness directions derived from final-turn activations of multi-turn attacks measured by median AUROC across layers.

Figure 5: Median cosine similarity between harmfulness directions derived from final-turn activations across attack frameworks

Figure 6: Top: cosine similarity between harmfulness directions derived from instruction tokens.

Figure 7: Cosine similarity between harmfulness and success/failure directions within each framework across layers.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Turn start	Turn end
Llama-3.1-8B-Instruct	<\|start_header_id\|> role <\|end_header_id\|>	<\|eot_id\|>
Qwen2.5-7B-Instruct	<\|im_start\|> role	<\|im_end\|>
Gemma-2-9B-it	<start_of_turn> role	<end_of_turn>

Appendix

Table 2: Chat template special tokens for each target model.

Figure 8: AUROC comparison of token-specific directions across conversation stages and attack frameworks. $v_{\text{inst}}$ represents harmfulness directions derived from harmful-versus-benign differences at the attacker instruction token, while $v_{\text{post}}$ represents success/failure directions derived from successful-versus-failed attack differences at the end-of-turn token. Values report median AUROC across layers.

Adversarial Attacks on LLMs

Adversarial attacks on large language models probe how crafted inputs, internal changes, and agent interactions bypass safety or disrupt reliable behavior.

See all 355 papers

Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models
Apr 23, 2026Naheed Rayhan, Sohely JahanLanguage Model Safety Evaluation Adversarial Attacks on LLMs
Jagannath University · University of Barishal
Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures
Aug 17, 2026Srinivasan Subramanian, Kazi Aminul Islam, Md. Abdullah Al Hafiz KhanLLM Safety LLM Safety Evaluation
Department of Computer Science, Kennesaw State University, Marietta, GA, USA
One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent
May 7, 2026Xinjie Shen, Rongzhe Wei, Peizhi Niu +6Language Model Safety Evaluation LLM Guardrails
Georgia Institute of Technology · University of Illinois Urbana-Champaign · UCSD +3

Apr 23, 2026J/K move · Enter open · S save

Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models

Naheed Rayhan, Sohely Jahan

Jagannath University · University of Barishal

Language Model Safety Evaluation Adversarial Attacks on LLMs Jailbreak Attacks Multi-Turn Jailbreak Attacks Black-Box Adversarial Attacks

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly integrated into sensitive workflows, raising the stakes for adversarial robustness and safety. This paper introduces Transient Turn Injection(TTI), a new multi-turn attack technique that systematically exploits stateless moderation by distributing adversarial intent across isolated interactions. TTI leverages automated attacker agents powered by large language models to iteratively test and evade policy enforcement in both commercial and open-source LLMs, marking a departure from conventional jailbreak approaches that typically depend on maintaining persistent conversational context. Our extensive evaluation across state-of-the-art models-including those from OpenAI, Anthropic, Google Gemini, Meta, and prominent open-source alternatives-uncovers significant variations in resilience to TTI attacks, with only select architectures exhibiting substantial inherent robustness. Our automated blackbox evaluation framework also uncovers previously unknown model specific vulnerabilities and attack surface patterns, especially within medical and high stakes domains. We further compare TTI against established adversarial prompting methods and detail practical mitigation strategies, such as session level context aggregation and deep alignment approaches. Our study underscores the urgent need for holistic, context aware defenses and continuous adversarial testing to future proof LLM deployments against evolving multi-turn threats.

Abstract

Adversarial Attacks on LLMs

Explore similar work

Transient Turn Injection: Exposing Stateless Multi-Turn Vulnerabilities in Large Language Models

Beyond Unsafe Detection: Counterfactually Anchored Evidence Attribution for Multi-Turn LLM Safety Failures

One Turn Too Late: Learning When to Intervene Against Multi-Turn Malicious Intent