cs.AISep 26, 2026
SaveLogical subspace in LLMs
Organizations: MIT · UPenn
Abstract
Recent work has identified a human brain network specialized for abstract formal reasoning (Kean et al., 2025). Does the same hold true in language models? To answer this question, we introduce the minimal viable subspace (MVS) method, which searches for the lowest-rank activation subspace at a layer that preserves task performance when everything outside that subspace is ablated. Using MVS, we demonstrate low-rank subspaces supporting logical inference on Gemma and Qwen models. Furthermore, these subspaces exhibit a clear dissociation from model capacities on other tasks, such that retaining these late logic subspaces preserves inference while impairing factual knowledge, working memory, cognitive control, and arithmetic. Conversely, ablating them reduces logical inference accuracy to chance while largely sparing these other capacities. Our results suggest a functionally localizable core machinery for logic akin to that in the human brain.
Figures & tables
Figure 1: Left: A human brain network for logical reasoning that selectively activates on logic tasks, but not on non-logic tasks ( Kean et al., 2025 ) . Right: This paper’s approach to network functional localization, which finds “logic subspaces” of activations such that ablating these subspaces destroys logical capabilities but preserves control tasks, and retaining these subspaces preserves logical capabilities and destroys abilities on control tasks. We report the averages over Qwen3-8B (circles) and Gemma4-12B (triangles) for ablating subspaces at layers 22 and 21 respectively. Dashed orange lines indicate accuracy in the intact network.
| Form | Fact cites | Fact | Conclusion | Validity |
|---|---|---|---|---|
| Modus ponens (MP) | antecedent | affirms | valid | |
| Denying the antecedent (DA) | antecedent | denies | invalid | |
| Affirming the consequent (AC) | consequent | affirms | invalid | |
| Modus tollens (MT) | consequent | denies | valid |
Table 1: The four conditional-inference forms for a rule , crossing whether the minor premise concerns the antecedent or consequent with whether it affirms or denies that proposition. Modus ponens and modus tollens are valid; denying the antecedent and affirming the consequent are invalid.
Figure 2: Task performance on Gemma4-12B and Qwen3-8B when retaining only the subspace found with MVS on logic using the projection ( 1 ). Our measure of performance is . A score of 1 indicates the fully intact model’s capabilities are preserved, while a score of 0 indicates chance-level accuracy. Note that logic tasks (including held-out formulations) are preserved by projection to the low-rank subspaces, but non-logic tasks are not (until the final layers, which correspond to a shared output channel between logic and the other tasks). This figure with fine-grained task info is available in Appendix A .
Figure 3: Probing each layer of Qwen3-8B and Gemma4-12B on the syllogistic reasoning tasks shows that logical features become linearly decodable with depth, with near-perfect decodability by layer 21 and layer 24, respectively.
Figure 4: A low-rank subspace is selectively necessary and sufficient for logical inference. Qwen3-8B was modified at with rank and Gemma-4-12B at with rank ; both use the learned for syllogisms by Eq. (2). Top: the intact model. Middle: ablating , . Bottom: retaining , Eq. (1). Bars are the mean over the two models and individual points indicate individual models ( Qwen3-8B, Gemma-4-12B); orange dashes give each task’s intact accuracy. Ablating drops all three syllogism sets to chance ( , , ) while factual knowledge, working memory, cognitive control and arithmetic performance is unchanged. Retaining reverses the effect, such that syllogisms are preserved ( , , ) and the controls fall to or below chance. PrOntoQA, considered to be a multi-step logic/deduction task, performance is affected by both interventions and is plotted separately, we return to this in the Discussion.
Figure 5: Ablating the logic subspaces at intermediate layers of Gemma and Qwen preserves language modeling ability but destroys logic. Retaining the subspaces preserves logic, but destroys language modeling ability. This shows the dissociation results presented in Figure 1 extend to language modeling, which does not have an accuracy score from 0 to 1 so it was not presented in that figure.
Figure 6: We plot the projection to the rank-2 logic subspace of the activations at Layer 22 of Qwen3-8B and Layer 22 of Gemma4-12B of syllogistic statements of various forms: Modus Ponens (MP), Denying the Antecedent (DA), Affirming the Consequent (AC), and Modus Tollens (MT). The activations for Qwen3-8B are linearly separable by validity. The activations for Gemma4-12B show cluster structure corresponding to the different logical forms. More plots are available in Figure 10 .
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Full results from Figure 2 for performance recovered on Qwen3-8B when retaining logic subspace at different ranks and layers.
Figure 8: Full results from Figure 2 for performance recovered on Gemma4-12B when retaining logic subspace at different ranks and layers.
Figure 9: Left (Qwen3-8B): The rank-2 logic subspace ablation of the last evaluated layer on Qwen hurts performance on MMLU with 1/2 output format, but not on MMLU with A/B output format. The percentage of correctly outputted answers for MMLU 1/2 drops to 0% for that layer, indicating that this subspace controls the output channel for the logic task, which is shared with the output channel for MMLU 1/2 but not for MMLU A/B. Right (Gemma-4-12B): At the last evaluated layer of gemma, the effect of ablating the logic subspace still shows a contrast between MMLU 1/2 and MMLU A/B, although it is not as dramatic as for Qwen in the left panel. The general decrease in accuracy on all formats might hypothetically correspond to a more abstract output channel for Gemma than for Qwen, which is shared across these formats.
Figure 10: Visualization of projection of the activations of Qwen3-8B and Gemma4-12B to the rank-2 logic subpaces at layer 22 and layer 21, respectively. We see that cluster structure emerges, corresponding to the different abstract underlying syllogisms.
| Family | Formulation | Rule, as stated (affirmative / negated) |
| if_then | material conditional | If the cube is little, then it is yellow. / …then it is not yellow. |
| fact_first | material conditional | (fact stated first) If the cube is little, then it is yellow. |
| reversed_if | material conditional | The cube is yellow if it is little. / The cube is not yellow if… |
| suppose_observe † | material conditional | Suppose that if the cube is little, then it is yellow. |
| whenever † | generic / habitual | Whenever the cube is little, it is yellow. / …it is not yellow. |
| reversed_whenever | generic / habitual | The cube is yellow whenever it is little. |
Table 2: The template families, grouped by how the rule is formulated. marks the three families held out from all optimization. Only the rule is shown; the fact and conclusion follow the argument form of Table 1 . The two event families use tensed verbs rather than adjectival predication.
More in this topic
LLM Reasoning
Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.
See all 931 papers