Are Causal Neuron Circuits Unique and Transferable?
Organizations: İletişim Yazılım
Abstract
Interpretability demonstrations often show that intervening on a group of neurons changes a model's answer. A recent example removes an 18-neuron group in Qwen3-32B and the model answers "Simba" instead of "Scar" to a two-hop Lion King riddle. Such a demonstration establishes that the group matters, but not how much of it is necessary, whether other groups could do the same, or whether the effect carries over to other inputs. We ask: when intervening on a group of neurons changes a model's answer, how much of that group is actually necessary? For every selected group we measure all 4,096 subsets of 12 MLP neurons. On a two-alternative capital task (Qwen3-0.6B), zero-ablating 12 selected neurons lowers the answer margin by 27% while random neurons barely change it, yet replacing the same neurons with their values from other questions also barely changes it: their activations are nearly constant across questions, and we find no evidence that they carry question-specific content. On two-hop relation chains, patching 12 neurons with activations from the first-hop question shifts the answer to the bridge entity (from Scar to Simba): the greedy answer starts with the bridge in 9/11 chains for Qwen3-1.7B, 9/11 for Llama-3.2-1B and 8/10 for Gemma-3-1B; random neurons never do. The full group is rarely necessary: a few neurons often suffice, and in many chains disjoint subsets each flip the answer on their own, in an exploratory analysis most often when the model is less confident. Group effects transfer to rephrasings of the same chain but are much weaker on other tested chains. We study neuron-level causal interventions and alternative subsets; we do not recover the full connectivity of circuits.
Figures & tables
| Model | Hop 1 | Hop 2 | Composed (phr. 1) | Composed (phr. 2) | All four correct | Included |
|---|---|---|---|---|---|---|
| Qwen3-1.7B | 20/20 | 20/20 | 12/20 | 14/20 | 12/20 | 11 |
| Llama-3.2-1B | 19/20 | 20/20 | 12/20 | 11/20 | 10/20 | 11 |
| Gemma-3-1B | 20/20 | 20/20 | 11/20 | 11/20 | 9/20 | 10 |
| Chain | Answer bridge | No intervention | P group patched | Z group zeroed |
|---|---|---|---|---|
| apple | Steve Jobs | Steve Jobs | Steve Jobs | Jobs |
| beatles | England Liverpool | England | Liverpool, England | Liverpool, England |
| eiffel | Paris France | Paris | France | France |
| lisbon | Atlantic Portugal | Atlantic Ocean | Portugal | The Atlantic Ocean |
| lk | Scar Simba | Scar | Simba | Simba |
| madrid | Spanish Spain | Spanish | Spain | Spanish |
| Qwen3-1.7B | Llama-3.2-1B | Gemma-3-1B | |
|---|---|---|---|
| Patching flips the answer | 6/7 | 6/7 | 6/7 |
| Generation starts with the bridge | 6/7 | 6/7 | 6/7 |
| Random neurons flip the answer | 0/7 | 0/7 | 0/7 |
| /m 0 , full group | 1.52 [1.23, 1.62] | 1.83 [1.17, 2.21] | 1.80 [1.23, 3.17] |
| Smallest flipping subset (neurons) | 4, 4, 4, 4, 5, 6 | 1, 2, 2, 2, 4, 8 | 1, 1, 2, 3, 4, 7 |
| Chains with disjoint flipping subsets | 1/7 | 4/7 | 5/7 |
| Chain | Composed question (phrasing 1) | Bridge | Answer | Included in |
|---|---|---|---|---|
| hp | What is the real name of the wizard who killed Harry Potter’s parents? | Voldemort | Tom | — |
| lk | Who killed the father of the main character of The Lion King? | Simba | Scar | Qwen3, Llama, Gemma |
| mona | In which country was the painter of the Mona Lisa born? | Leonardo | Italy | Qwen3 |
| eiffel | What is the capital of the country where the Eiffel Tower is located? | France | Paris | Qwen3, Llama, Gemma |
| madrid | What language is spoken in the country whose capital is Madrid? | Spain | Spanish | Qwen3, Llama, Gemma |
| tokyo | What is the currency of the country where Tokyo is located? | Japan | yen | Llama |
Circuits in Language Models
Circuits in language models are interacting internal components that implement behaviors and can be traced with causal interventions.
See all 129 papers