Causal Interventions in Language Models

Overview

This field asks which internal states, features, heads, neurons, or context elements actually influence a language model’s decisions, rather than merely correlate with them. Researchers use counterfactual prompt and evidence edits, activation patching and ablation, causal tracing, sparse-autoencoder features, and inference-time steering or weight edits to test mechanisms and control behavior. Applications include checking whether chain-of-thought and retrieved evidence drive answers, and modifying refusal, hallucination, bias, tool use, or reasoning while preserving unrelated capabilities. Results show that readable representations are often not causally used or easily steered; effects vary by layer, task, model, and context, while redundancy and collateral effects make reliable, selective interventions a continuing challenge.

Latest papers

CardsList

532 papers55 papers in the last four weeks, down 37% on the four weeks before. 0.4% of all new papers.

  1. BOS Summary: The First Token Matters

    Nov 17, 2026Hui Zhu, Saleh Zare Zade, Rafi Ibn Sultan +3Attention Intervention in VLMsAttention Sinks

  2. What MoE Routing Controls: Causal Interventions on Expert Trajectories

    Nov 17, 2026Mukund Athmaram, Manoj Rao, Liang MaMixture-of-Experts Language ModelsLanguage Model Steering

  3. Pretty Linear Liars: Unregularized Probes Favor Intervention-Unfaithful Features

    Nov 16, 2026Karl Vilhelmsson Emneby, Julian Yocum, Cameron AllenLinear ProbingMechanistic Interpretability

  4. Where Is Rank-1 DiM Refusal Intervenable in Multimodal LM Derivatives?

    Nov 15, 2026Chansu Han, Seok-Hwan Choi, Takeshi TakahashiMLLM SafetyCausal Interventions in Language Models

  5. Right Token, Right Time: From Per-Token Gating to Trajectory-Aware Activation Steering

    Nov 13, 2026Nima Eshraghi, Lovedeep Gondara, Sagarika Suresh +3Language Model SteeringActivation Steering

  6. Sanity Checks for Attention Surgery: Null Models and Matched Interventions for Representation-Motivated Design

    Nov 10, 2026Basavaraj R M, Aryan Bharat KumarAttention MechanismsAttention Head Analysis

  7. SWEET: Mitigating Spurious Feature Reliance in LLM Reasoning via Weight Editing

    Nov 7, 2026Zhizhang Fu, Lucas Zhao, Michalis Korakakis +3Faithfulness of Language Model ReasoningLLM Interpretability

  8. AttentionSwitch: Jailbreak Defense for LLMs via Joint Learning of Sparse Attention-Head Selection and Scaling

    Nov 5, 2026Doniyorkhon Obidov, Honggang Yu, Xiaolong Guo +1Jailbreak DefenseLLM Safety

  9. The Intrusion Pathway: How Non-Evidential Context Degrades Language Model Performance

    Nov 5, 2026Dikshant Kukreja, Gautam Gupta, Avni Mittal +8Language Model RobustnessMechanistic Interpretability

  10. Localization Does Not Imply Steerability: A Mechanistic Study of Sign Errors in LLM Mathematics

    Nov 5, 2026Shradha Agarwal, Tariq J., Deepak RajbharLLM InterpretabilityCircuits in Language Models

  11. When Every Choice Is Correct: A Causal Study of Numerical Preference in Language Models

    Nov 4, 2026Hongnian Huang, Hongliang LiuMechanistic InterpretabilityCircuits in Language Models

  12. Beyond Accuracy: What Language-Subspace Interventions Capture, Remove, and Improve

    Nov 3, 2026Shaanil Punglia, Dadhichi Telwadkar, Vaibhav Rathore +2Causal Interventions in Language ModelsCross-Lingual Reasoning in Language Models

  13. Whose Effect Is It? How Objectives Inflate Learned Subspace Interventions

    Nov 3, 2026Toygar Tanyel, Serkan AyvazMechanistic InterpretabilityExplainability Evaluation

  14. TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport

    Nov 3, 2026Chuancheng Shi, Shangze Li, Wenhua Wu +5LLM SafetyLLM Safety Alignment

  15. When Should an LLM Steering Edit Depend on the Input?

    Nov 3, 2026Rachael Harkness, Konrad RawlikLanguage Model SteeringActivation Steering