cs.LGNov 10, 2026

Are Causal Neuron Circuits Unique and Transferable?

Authors: Ömer Tekin Yavuz

Organizations: İletişim Yazılım

Abstract

Interpretability demonstrations often show that intervening on a group of neurons changes a model's answer. A recent example removes an 18-neuron group in Qwen3-32B and the model answers "Simba" instead of "Scar" to a two-hop Lion King riddle. Such a demonstration establishes that the group matters, but not how much of it is necessary, whether other groups could do the same, or whether the effect carries over to other inputs. We ask: when intervening on a group of neurons changes a model's answer, how much of that group is actually necessary? For every selected group we measure all 4,096 subsets of 12 MLP neurons. On a two-alternative capital task (Qwen3-0.6B), zero-ablating 12 selected neurons lowers the answer margin by 27% while random neurons barely change it, yet replacing the same neurons with their values from other questions also barely changes it: their activations are nearly constant across questions, and we find no evidence that they carry question-specific content. On two-hop relation chains, patching 12 neurons with activations from the first-hop question shifts the answer to the bridge entity (from Scar to Simba): the greedy answer starts with the bridge in 9/11 chains for Qwen3-1.7B, 9/11 for Llama-3.2-1B and 8/10 for Gemma-3-1B; random neurons never do. The full group is rarely necessary: a few neurons often suffice, and in many chains disjoint subsets each flip the answer on their own, in an exploratory analysis most often when the model is less confident. Group effects transfer to rephrasings of the same chain but are much weaker on other tested chains. We study neuron-level causal interventions and alternative subsets; we do not recover the full connectivity of circuits.

Figures & tables

Explore similar work

CardsList
  1. How Much Do Circuits Tell Us? Measuring the Consistency and Specificity of Language Model Circuits

    May 8, 2026Michael Li, Nishant SubramaniCircuit DiscoveryMechanistic Interpretability

  2. Causal Neural Probabilistic Circuits

    Mar 2, 2026Weixin Chen, Han ZhaoConcept Bottleneck ModelsCausal Intervention

  3. Faithfulness to Refusal: A Causal Audit of Neuron Selectors

    Jul 6, 2026Ananth Eswar, Pratinav Seth, Utsav Avaiya +1Transformer InterpretabilityFeature Attribution