Causal Interventions in Language Models

This field asks which internal states, features, heads, neurons, or context elements actually influence a language model’s decisions, rather than merely correlate with them. Researchers use counterfactual prompt and evidence edits, activation patching and ablation, causal tracing, sparse-autoencoder features, and inference-time steering or weight edits to test mechanisms and control behavior. Applications include checking whether chain-of-thought and retrieved evidence drive answers, and modifying refusal, hallucination, bias, tool use, or reasoning while preserving unrelated capabilities. Results show that readable representations are often not causally used or easily steered; effects vary by layer, task, model, and context, while redundancy and collateral effects make reliable, selective interventions a continuing challenge.