Causal Interventions in Language Models

Follow this topic Ask about this topic

This field asks which internal states, features, heads, neurons, or context elements actually influence a language model’s decisions, rather than merely correlate with them. Researchers use counterfactual prompt and evidence edits, activation patching and ablation, causal tracing, sparse-autoencoder features, and inference-time steering or weight edits to test mechanisms and control behavior. Applications include checking whether chain-of-thought and retrieved evidence drive answers, and modifying refusal, hallucination, bias, tool use, or reasoning while preserving unrelated capabilities. Results show that readable representations are often not causally used or easily steered; effects vary by layer, task, model, and context, while redundancy and collateral effects make reliable, selective interventions a continuing challenge.

Latest papers

489 papers83 papers in the last four weeks, up 22% on the four weeks before. 0.5% of all new papers.

Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models
Jan 12, 2026Zhenghao He, Guangzhi Xiong, Bohan Liu +2LLM Interpretability CoT Reasoning
University of Virginia, USA
Mechanistic Knobs in LLMs: Retrieving and Steering High-Order Semantic Features via Sparse Autoencoders
Jan 6, 2026Ruikang Zhang, Shuo Wang, Qi SuLLM Interpretability Activation Steering
Peking University, Beijing, China
SGM: Safety Glasses for Multimodal Large Language Models via Neuron-Level Detoxification
Dec 17, 2025Hongbo Wang, AprilPyone MaungMaung, Isao EchizenMLLM Safety Multimodal Robustness
1Graduate School of Information Science and Technology, The University of Tokyo, Japan · 2Information and Society Research Division, National Institute of Informatics, Japan · 3The Graduate University for Advanced Studies, SOKENDAI, Japan
D-STEER: Two-Way Control of DPO-Induced Safety Behavior in Activation Space
Dec 3, 2025Samarth Raina, Saksham Aggarwal, Amit Dhanda +3Causal Interventions in Language Models Activation Steering
IIIT Delhi · Microsoft · Apple (USA) +2
LIFT: Likelihood-Based Intervention Faithfulness Training
Sep 10, 2025Anand Swaroop, Maheep Chaudhary, Kevin Zhu +2CoT Faithfulness Faithfulness of Language Model Explanations
Algoverse AI Research
LLMs Encode Harmfulness and Refusal Separately
Jul 16, 2025Jiachen Zhao, Jing Huang, Zhengxuan Wu +2LLM Interpretability LLM Refusal Behavior
Northeastern University · Stanford University
When Can Large Reasoning Models Save Thinking? Mechanistic Analysis of Behavioral Divergence in Reasoning
May 21, 2025Rongzhi Zhu, Yi Liu, Jiancheng Wang +6Overthinking in Language Models LLM Interpretability
State Key Laboratory for Novel Software Technology, Nanjing University, China · China Unicom Software Research Institute, China · National Institute of Healthcare Data Science, Nanjing University, China +1
Quantifying Symptom Causality in Clinical Decision Making: An Exploration Using CausaLM
Mar 25, 2025Alexander Romanus, Mehul Shetty, Connor JordanClinical Decision-Making Causal Interpretability
Verbosity Tradeoffs and the Impact of Scale on the Faithfulness of LLM Self-Explanations
Mar 17, 2025Noah Y. Siegel, Nicolas Heess, Maria Perez-Ortiz +1Faithfulness of Language Model Explanations LLM Interpretability
Google DeepMind · Centre for AI, University College London · Imperial College London +1

Jan 12, 2026J/K move · Enter open · S save

Triggering Chain-of-Thought via Latent Feature Interventions in Large Language Models

Zhenghao He, Guangzhi Xiong, Bohan Liu, Sanchit Sinha+1

University of Virginia, USA

LLM Interpretability CoT Reasoning Causal Interventions in Language Models LLM Reasoning Latent CoT Reasoning

Open paper AskPDF arXiv

Chain-of-Thought (CoT) prompting often improves the reasoning performance of large language models (LLMs), but the internal signal that triggers this behavior remains poorly understood. Leveraging the sparse features captured by Sparse Autoencoders (SAEs), we propose a systematic framework to analyze and intervene on the internal representations of LLMs, identifying a small set of latent features that are linked to reasoning behavior and can be causally tested through targeted intervention. Across multiple model families and reasoning benchmarks, we show that steering one or a small number of reasoning-related latent features can substantially induce reasoning behavior without explicit CoT prompting, achieving accuracy comparable to CoT. We further show that the identified features are not tied to particular wording patterns or verbosity, and confirm their causal role in reasoning through suppression experiments that impair performance even under CoT prompting. These results suggest that CoT prompting activates specific latent features to trigger reasoning, and that targeted intervention on these features offers an alternative pathway to elicit efficient reasoning behavior without explicit CoT prompting. Code is available at https://github.com/Zhenghao-He/LatentCoT.