cs.CLApr 1, 2026
SaveFriends and Grandmothers in Silico: Localizing Entity Cells in Language Models
Organizations: Mentaleap · Independent Researcher · Tel Aviv University
Abstract
How do language models retrieve entity-specific facts from their parameters? We investigate this question by searching for sparse, entity-selective MLP neurons - which we call entity cells, by analogy to the "grandmother cell" hypothesis in neuroscience - and testing whether they play a causal role in factual recall. We localize candidate entity cells by ranking MLP neurons for activation consistency across varied prompts about the same entity, applying this procedure across seven models on a curated subset of PopQA. In all models, localized neurons cluster predominantly in early layers, an empirical pattern not imposed by the architecture. Using Qwen2.5-7B base as a model organism, we find the clearest causal evidence: suppressing a localized cell selectively erases recall for its matched entity while leaving others intact, and activating a single cell is sufficient to recover correct knowledge for most entities - even when the entity is absent from the context. The same cells are recovered under aliases, acronyms, misspellings, and multilingual surface forms, and remain stable through instruction tuning, suggesting they encode canonical entity identity rather than surface token patterns. Causal signals vary across model families, pointing to architectural differences in how entity knowledge is organized. These findings offer concrete, interpretable access points for understanding, controlling, and correcting factual knowledge in language models, and draw a surprising empirical parallel to longstanding questions in neuroscience about sparse coding of concepts.
Figures & tables
Figure 1: We identify sparse, entity-selective MLP neurons, termed entity cells , that act as stable anchors for factual retrieval in Qwen2.5-7B. Concentrated primarily in early layers (0–5), these cells provide access to canonical identity representations that are robust to aliases, misspellings, and multilingual variants. These neurons serve as causally actionable access points: suppressing them induces entity-specific amnesia, while activating a single localized neuron is often sufficient to steer the model toward entity-consistent factual recall. Across the other six models in our suite, early-layer candidates also appear, though the causal validation is weaker.
Figure 2: Layer of the top localized cell for each PopQA-200 entity in Qwen2.5-7B base (n=200). Similar early-layer concentration is observed across other tested models; see Appendix G .
Figure 3: Entity-specific amnesia under negative ablation for the localized Obama cell (L2-N10941). Target (Obama) recall drops substantially as decreases, while control (Trump) remains near baseline.
Figure 4: Controlled injection at the placeholder token X , evaluated on instances where the entity-present prompt is already correct under pass@5 (109 examples). Mean-entity initialization and wrong-cell injection are control conditions; correct-cell injection shows the expected directional gain.
Figure 5: Variant robustness for “Barack Obama”: most spelling and phrasing perturbations keep the same localized cell (L2-N10941).
Figure 6: Acronym robustness (FBI): variants localize to the same top-ranked cell (L2-N11955).
Figure 7: Multilingual robustness (Paris): variants localize to the same top-ranked cell (L1-N231).
Appendix figures & tables21 assets
Supplementary material from the paper’s appendix.
Appendix
| Use | Template |
|---|---|
| PopQA QA wrapper | Question: <q>\nAnswer: |
| Generic baselines | <statement fragment> |
| Localization probes | The <attribute> of <entity> |
| Injection/ablation | Fact: the <relation> of <entity>: |
| Factual modification | The spouse of <entity> is named |
Table 1: Prompt templates used in this work.
| ID | Prompt (Barack Obama) |
|---|---|
| A1 | The name of the wife of Barack Obama is |
| A2 | When Barack Obama was president, his wife’s name was |
| A3 | Barack Obama is married to |
| A4 | The spouse of Barack Obama is named |
| P1 | Barack Obama was born in Hawaii |
| P2 | The political party of Barack Obama is Democratic |
Table 2: Attack prompts (A1–A4) and preservation prompts (P1–P6) used for factual modification. Preservation prompts include an expected next token.
Figure 8: Factual modification via latent steering (single-case study). Top: spouse prompts (A1–A4; Table 2 ) before/after steering toward a target completion. Bottom: preservation prompts (P1–P6), reported as steered/base ratios for expected next tokens.
Figure 9: Qwen2.5-7B-Instruct replication of Figure 2 . As in the main-paper Qwen2.5-7B base result, top localized cells remain concentrated in early layers, indicating that the layer profile is largely preserved under instruction tuning.
Figure 10: Qwen2.5-7B-Instruct replication of Figure 3 . Negative ablation of the localized entity cell again causes a strong drop for the target entity while leaving the control entity comparatively stable, supporting preservation of the same causal pattern after post-training.
Figure 11: Qwen2.5-7B-Instruct replication of Figure 4 . On the trusted set, correct-cell injection again outperforms both the mean-entity initialization and wrong-cell controls, indicating that the same localized cells remain causally useful after instruction tuning.
Figure 12: Qwen2.5-7B-Instruct replication of Figure 5 . Most variants of “Barack Obama” preserve the same top cell as in the base model, indicating that the localized handle remains stable under post-training.
Figure 13: Qwen2.5-7B-Instruct replication of Figure 6 . The acronym and full form again localize to the same top-ranked cell, consistent with preservation of the underlying entity-cell mapping.
Figure 14: Qwen2.5-7B-Instruct replication of Figure 7 . The same top cell is recovered across multiple scripts for “Paris”, suggesting that cross-script entity access also survives instruction tuning.
Figure 15: Qwen3-8B within-family replication of Figure 2 . As in the main-paper Qwen2.5 result, top localized cells remain concentrated in early layers, suggesting that the coarse localization pattern persists within the Qwen family.
Figure 16: Qwen3-8B within-family replication of Figure 3 , shown on an alternative London/Paris probe because it yields cleaner separation than the default Obama/Trump case in this model. Negative ablation still preferentially suppresses the target-entity curve, but the effect is less clean than in Qwen2.5.
Figure 17: Qwen3-8B within-family replication of Figure 4 . Correct-cell injection improves over the control conditions, but the separation is weaker and less consistent than in Qwen2.5, matching the more mixed within-family result described in the text.
Figure 18: Qwen3-8B within-family replication of Figure 5 . Most variants recover closely related early-layer cells, though the match is less clean than in Qwen2.5-7B-Instruct.
Figure 19: Qwen3-8B within-family replication of Figure 6 . The acronym and expanded form still localize to closely aligned cells, supporting partial preservation of the entity-cell map within the Qwen family.
Figure 20: Qwen3-8B within-family replication of Figure 7 . Cross-script forms continue to recover similar top cells, though the pattern is noisier than in Qwen2.5.
| Entity | Layer | Neuron | Notes |
|---|---|---|---|
| Obama | 0 | 2883 | localized (injection weak in this subset) |
| Trump | 3 | 9290 | localized; injection success |
| Paris | 2 | 10547 | localized |
| London | 0 | 3037 | localized; strong ablation drop |
| Beijing | 4 | 5431 | localized; strong ablation drop |
| Tokyo | 3 | 223 | localized; ablation drop |
Table 3: Representative Qwen3-8B base entity cells localized by stability. Notes summarize small-subset causal checks (Appendix F ).
Figure 21: Cross-model localization depth profiles (2 2) for four non-Qwen-family models, formatted for single-column display. Each panel shows the distribution of top-neuron layers over PopQA-200 entities for one model. Relative to the main-paper localization result in Figure 2 , these families are generally broader and deeper.
Figure 22: Controlled injection on trustworthy cells (top- , alpha search). Left: causal separation , where probabilities are normalized by the entity-present prompt. Right: entity-level success rate, defined as the fraction of trustworthy entities that satisfy all three criteria: , improvement over no-injection , and improvement over wrong-cell injection .
Figure 23: OLMo-7B cross-family replication of Figure 4 . Among the non-Qwen models, OLMo shows the strongest positive result: correct-cell injection produces the clearest separation from the control conditions on the trusted set.
Figure 24: OLMo-7B cross-family replication of Figure 3 . Negative ablation reduces the target-entity curve, but the control curve is also affected, so this is not the same clean entity-specific amnesia pattern seen in Qwen2.5. This suggests that OLMo’s localized cells may participate in retrieval differently, or less selectively, than the Qwen-family cells.
| Model | Trustworthy | Injection | Form-Robustness |
|---|---|---|---|
| OLMo-7B | 37 | 23/37 | 30% |
| Llama-3.1-8B | 40 | 5/40 | 50% |
| Mistral-7B | 40 | 5/40 | 40% |
| OpenLLaMA-7B | 33 | 12/33 | 30% |
Table 4: Cross-family summary over PopQA-200 entities. Trustworthy Cells : entities retained by the Finding 2 amnesia trust filter. Knowledge Injection : top- success among trustworthy entities. Surface-Form Robustness : exact top-cell match rate across variant, acronym, and multilingual probes (10 attempts per probe).