The Geometry of Conceptors in Neural-Activation Spaces
Organizations: New York University · University of Pennsylvania · University of Cincinnati · Linguistic Data Consortium, University of Pennsylvania
Abstract
Conceptors represent concepts as graded collections of directions in neural-activation space rather than as single vectors. We develop a geometric account of how these representations depend on the contrast used to define a concept: opposing values in bipolar fitting, or presence and absence in unipolar fitting. We derive an exact decomposition of a centered pooled conceptor into an average pole filter, a mean-separation update, and a covariance-shape update. Experiments across five language models on sentiment, political leaning and stance, and depression expression show that both updates improve reconstruction of held-out contrasts. Additional directions preserve variation beyond the mean-difference line, and changing the comparator changes the benefit of these directions. Shape, size, and overlap each show up downstream: in amplification experiments, bipolar fits raise evidence for both poles more consistently than unipolar fits; conceptor size (the quota) correlates positively with layerwise probe performance; and cross-concept overlap quantifies interference under subspace removal, linking shared directions to Boolean composition. In an additional generation evaluation, conceptor filters achieve higher target-expression win rates against an unsteered reference than three mean-based controls. Together, these findings explain how fitting choices shape conceptor geometry and support bipolar fitting when the objective is to amplify both opposing poles.
Figures & tables
| Own mean | Shared source center | ||||||
|---|---|---|---|---|---|---|---|
| Model | Data | Pool | Pool | ||||
| Gemma 2 2B | Sentiment | 75.5 | 52.3 | 48.4 | 75.4 | 78.1 | 80.2 |
| Pol. Stance | 51.7 | 49.8 | 50.8 | 51.7 | 50.1 | 51.0 | |
| Depression | 69.8 | 51.1 | 52.7 | 69.8 | 70.6 | 69.6 | |
| Gemma 2 9B | Sentiment | 74.3 | 54.4 | 52.6 | 74.3 | 76.3 | 77.9 |
| Pol. Stance | 48.5 | 47.7 | 47.8 | 48.5 | 48.3 | 48.2 | |
| Data | Mean line | ||
|---|---|---|---|
| Sentiment ( ) | 35.7 / 36.4 | 61.1 / 61.1 | 68.3 / 68.4 |
| Pol. Stance ( ) | 23.2 / 24.7 | 35.1 / 37.6 | 50.7 / 52.3 |
| Depression ( ) | 40.8 / 36.5 | 49.7 / 44.8 | 62.1 / 58.7 |
| Concept pair | Gemma 2 2B | Gemma 2 9B | Qwen 2.5 3B | Llama 3.1 8B | OLMo 3 7B |
|---|---|---|---|---|---|
| Sentiment–Depression | 0.321 | 0.300 | 0.332 | 0.318 | 0.221 |
| Sentiment–Pol. Stance | 0.154 | 0.134 | 0.169 | 0.096 | 0.035 |
| Depression–Pol. Stance | 0.163 | 0.143 | 0.182 | 0.108 | 0.042 |
| Bipolar fit | Unipolar fit | |||||
|---|---|---|---|---|---|---|
| Method | Positive pole | Opposite pole | Both improve | Positive pole | Opposite pole | Both improve |
| Mean addition | +0.2942 | -0.2942 | 0/30 | +0.0550 | -0.0550 | 0/30 |
| amplification | +0.0018 | +0.0002 | 10/30 | +0.0177 | -0.0153 | 2/30 |
| amplification | +0.0301 | +0.0322 | 30/30 | +0.0177 | -0.0083 | 7/30 |
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing | Mean |
| Full conceptor filter | 55.5 | 47.2 | 40.2 | 32.5 | 48.2 | 58.8 | 47.1 |
| Rank-one conceptor filter | 58.4 | 52.5 | 41.3 | 34.8 | 52.0 | 61.4 | 50.1 |
| Fixed mean-difference addition | 16.3 | 12.9 | 13.7 | 10.7 | 13.7 | 13.3 | 13.4 |
| Pooled-mean addition | 15.0 | 15.8 | 15.4 | 11.7 | 12.9 | 15.6 | 14.4 |
| Mean-line amplification | 11.0 | 11.0 | 13.5 | 8.8 | 10.9 | 15.5 | 11.8 |
| Eight-shot ICL | 79.8 | 86.7 | 45.8 | 63.5 | 91.3 | 83.7 | 75.1 |
Appendix figures & tables36 assets
Supplementary material from the paper’s appendix.
Appendix
| Collection | Concept | Formulations | Texts |
|---|---|---|---|
| BipolarConcepts | Sentiment | Positive; negative | 200 |
| Pol. Stance | Left; right | 200 | |
| Depression expression | Depression; counter-expressions | 200 | |
| ConceptVariants | Sentiment | Positive; negative; descriptive neutral; attitudinal neutral | 96 |
| Pol. Stance | Left; right; descriptive neutral; undecided | 96 | |
| Depression expression | Depression; ordinary functioning; explicit absence; flourishing; recovery | 120 |
| Benchmark | Probe target | Training / evaluation |
|---|---|---|
| SST-2 ( Socher et al., 2013 ) | Positive / negative sentiment | Train / validation |
| Rotten Tomatoes ( Pang & Lee, 2005 ) | Positive / negative sentiment | Train / test |
| TweetEval sentiment ( Barbieri et al., 2020 ) | Positive / negative sentiment | Train / test |
| Hyperpartisan News ( Kiesel et al., 2019 ) | Hyperpartisan / mainstream | Train / validation |
| Reddit Depression ( thePixel42, n.d. ) | Released binary proxy label | Train / test |
| Cross-cultural Reddit ( Rai et al., 2025 ) | Mental-health proxy label | Existing train / test |
| Collection | Group | Texts | Mean words | 95th pct. | Max. |
|---|---|---|---|---|---|
| ConceptVariants | All concepts | 312 | 10.0 | 14.0 | 16 |
| ConceptGroups | |||||
| Sentiment | Amazon | 200 | 11.5 | 26.0 | 30 |
| IMDb | 600 | 14.3 | 33.0 | 61 | |
| Yelp | 600 | 11.0 | 23.0 | 32 | |
| Pol. Stance | Abortion | 240 | 16.4 | 24.0 | 30 |
| Collection | Gemma median / max. | Qwen median / max. |
|---|---|---|
| ConceptVariants | 12 / 19 | 11 / 18 |
| ConceptGroups : Sentiment | 14 / 82 | 13 / 78 |
| ConceptGroups : Pol. Stance | 25 / 48 | 24 / 50 |
| ConceptGroups : Depression | 9 / 16 | 8 / 14 |
| Concept | Recorded positive pole | Recorded negative pole |
|---|---|---|
| Sentiment | I feel great today. | I feel bad today. |
| Pol. Stance | I support student debt relief. | I oppose student debt cancellation. |
| Depression | Nothing interests me. | Everything interests me. |
| Concept | Formulation | Matched-topic example |
|---|---|---|
| Sentiment | Positive | I found the film wonderfully engaging. |
| Negative | I found the film painfully dull. | |
| Descriptive neutral | I watched the film in the evening. | |
| Attitudinal neutral | I had no strong feelings about the film. | |
| Pol. Stance | Left | Regarding health insurance policy, I favor expanding publicly funded health coverage. |
| Right | Regarding health insurance policy, I favor relying more on privately purchased health coverage. |
| Review domain | Positive | Negative |
|---|---|---|
| Amazon | Battery life is also great! | Awkward to use and unreliable. |
| IMDb | This is such a fun and funny movie. | The problem was the script. |
| Yelp | The atmosphere here is fun. | The manager was the worst. |
| Facet | Symptom expression | Ordinary / positive counterpart |
|---|---|---|
| Appetite | I have no appetite and food feels like a chore. | I enjoy my meals and eat well. |
| Burden | I feel like a burden to everyone around me. | I feel valued by the people around me. |
| Concentration | I can’t concentrate on anything anymore. | I concentrate easily on what I’m doing. |
| Energy | I feel drained and tired all the time. | I feel energetic and lively all the time. |
| Hopelessness | The future looks completely bleak to me. | I look forward to the future with hope. |
| Interest | Nothing interests me anymore. | I’m genuinely interested in lots of things. |
| Topic | Favor | Against |
|---|---|---|
| Abortion | So prolifers are against health rights? | #ProLifeYouth know that life begins at conception. |
| Atheism | god of the gaps is not evidence #next | God is so good! All the praises go up to Him! |
| Climate change | RT @user 700 people dead in Pakistan due to heatwave. | @user ice cracking in the summer?? SO ALARMING. #climatechangehoax |
| Feminism | FINALLY A WOMEN RUNNING FOR PRESIDENT | Feminism is nothing more than a cult that closes ranks whenever someone dares question them. |
| Hillary Clinton | RT @user Hillary for the win | Hillary Clinton for Ambassador to ISIS. |
| Model | Blocks | Midpoint | Final | |
|---|---|---|---|---|
| Gemma-2-2B-IT | 26 | 2,304 | 13 | 25 |
| Gemma-2-9B-IT | 42 | 3,584 | 21 | 41 |
| Qwen-2.5-3B-Instruct | 36 | 2,048 | 18 | 35 |
| Llama-3.1-8B-Instruct | 32 | 4,096 | 16 | 31 |
| OLMo-3-7B-Instruct | 32 | 4,096 | 16 | 31 |
| Data | Site | Full | ||||
|---|---|---|---|---|---|---|
| Sentiment | Mid | 0.1 | 0.1344 | 0.1052 | 0.0747 | 0.0536 |
| Mid | 1 | 0.2489 | 0.1466 | 0.1679 | 0.0984 | |
| Mid | 10 | 0.4637 | 0.2689 | 0.4284 | 0.2460 | |
| Final | 0.1 | 0.1964 | 0.1797 | 0.1262 | 0.1139 | |
| Final | 1 | 0.2792 | 0.2347 | 0.2166 | 0.1855 | |
| Final | 10 | 0.4805 | 0.3584 | 0.4376 | 0.3294 |
| Data | Site | Full | ||||
|---|---|---|---|---|---|---|
| Sentiment | Mid | 0.1 | 0.1573 | 0.1262 | 0.0987 | 0.0756 |
| Mid | 1 | 0.2573 | 0.1626 | 0.1878 | 0.1214 | |
| Mid | 10 | 0.4398 | 0.2722 | 0.4127 | 0.2545 | |
| Final | 0.1 | 0.2506 | 0.2315 | 0.1750 | 0.1597 | |
| Final | 1 | 0.3274 | 0.2820 | 0.2620 | 0.2298 | |
| Final | 10 | 0.5093 | 0.3922 | 0.4666 | 0.3640 |
| Data | Site | Full | ||||
|---|---|---|---|---|---|---|
| Sentiment | Mid | 0.1 | 0.0291 | 0.0182 | 0.0218 | 0.0137 |
| Mid | 1 | 0.0584 | 0.0411 | 0.0552 | 0.0388 | |
| Mid | 10 | 0.0783 | 0.0733 | 0.0780 | 0.0730 | |
| Final | 0.1 | 0.1675 | 0.1545 | 0.1036 | 0.0937 | |
| Final | 1 | 0.2213 | 0.1918 | 0.1679 | 0.1461 | |
| Final | 10 | 0.3924 | 0.3040 | 0.3422 | 0.2678 |
| Data | Site | Pooled | ||||
|---|---|---|---|---|---|---|
| Sentiment | Mid | 0.1 | 0.1011 | 0.0786 | 0.0695 | 0.0540 |
| Sentiment | Mid | 1 | 0.1687 | 0.1080 | 0.1437 | 0.0931 |
| Sentiment | Mid | 10 | 0.2460 | 0.1980 | 0.2420 | 0.1947 |
| Sentiment | Final | 0.1 | 0.1698 | 0.1567 | 0.1096 | 0.0982 |
| Sentiment | Final | 1 | 0.1985 | 0.1765 | 0.1436 | 0.1274 |
| Sentiment | Final | 10 | 0.3237 | 0.2511 | 0.2677 | 0.2126 |
| Data | Site | Pooled | ||||
|---|---|---|---|---|---|---|
| Sentiment | Mid | 0.1 | 0.3104 | 0.2661 | 0.1887 | 0.1477 |
| Sentiment | Mid | 1 | 0.3352 | 0.2714 | 0.2136 | 0.1654 |
| Sentiment | Mid | 10 | 0.4719 | 0.3207 | 0.3575 | 0.2470 |
| Sentiment | Final | 0.1 | 0.2560 | 0.2346 | 0.1773 | 0.1581 |
| Sentiment | Final | 1 | 0.2799 | 0.2479 | 0.2078 | 0.1830 |
| Sentiment | Final | 10 | 0.4007 | 0.3250 | 0.3413 | 0.2814 |
| Ablated projector | Pooled wins | Pooled losses | Mean gain (pp) |
|---|---|---|---|
| 84 | 6 | ||
| 77 | 13 | ||
| 90 | 0 | ||
| All comparisons | 251 | 19 |
| Denoising loss | Readout BA (%) | ||||
|---|---|---|---|---|---|
| Model | Data | Hard | Conceptor | Mean update | Pooled |
| Gemma-2B | Sentiment | 0.950 | 0.600 | 94.6 | 95.4 |
| Pol. Stance | 0.811 | 0.553 | 58.2 | 57.3 | |
| Depression | 0.845 | 0.638 | 98.5 | 96.9 | |
| Gemma-9B | Sentiment | 0.964 | 0.608 | 95.0 | 94.7 |
| Pol. Stance | 0.833 | 0.567 | 64.3 | 59.5 | |
| Model | Comparator | Mean line | Gap | |
|---|---|---|---|---|
| Gemma-2B | Flourishing | 22.0 [13.1, 31.5] | 51.2 [41.0, 53.9] | +29.1 |
| Ordinary functioning | 30.4 [11.6, 53.2] | 55.4 [42.9, 60.5] | +24.9 | |
| Explicit absence | 68.0 [48.0, 83.3] | 74.8 [63.6, 84.6] | +6.9 | |
| Gemma-9B | Flourishing | 20.2 [10.5, 34.2] | 52.8 [43.5, 55.1] | +32.6 |
| Ordinary functioning | 32.7 [12.7, 57.8] | 57.6 [44.3, 64.6] | +24.9 | |
| Explicit absence | 68.6 [46.7, 85.5] | 75.7 [63.4, 87.0] | +7.0 |
| Model | Comparator | Mean line | Gap | |
|---|---|---|---|---|
| Gemma-2B | Negative | 40.8 [27.3, 44.8] | 45.4 [37.1, 47.3] | +4.6 |
| Descriptive neutral | 37.3 [19.9, 47.6] | 50.2 [41.0, 55.4] | +13.0 | |
| Attitudinal neutral | 75.6 [65.4, 82.2] | 79.1 [71.4, 84.2] | +3.4 | |
| Gemma-9B | Negative | 35.1 [19.8, 39.5] | 38.3 [23.1, 42.9] | +3.1 |
| Descriptive neutral | 42.0 [22.4, 53.4] | 53.0 [42.2, 59.5] | +11.0 | |
| Attitudinal neutral | 79.6 [69.6, 85.9] | 82.0 [74.3, 87.1] | +2.4 |
| Model | Comparator | Mean line | Gap | |
|---|---|---|---|---|
| Gemma-2B | Right | 6.4 [1.1, 13.3] | 14.1 [8.6, 21.7] | +7.8 |
| Descriptive neutral | 65.1 [59.3, 66.7] | 70.5 [66.3, 71.2] | +5.4 | |
| Undecided | 63.8 [56.8, 66.2] | 69.0 [64.2, 70.2] | +5.2 | |
| Gemma-9B | Right | 8.5 [1.7, 17.1] | 15.9 [9.5, 25.8] | +7.4 |
| Descriptive neutral | 57.3 [50.3, 59.1] | 65.6 [60.6, 65.9] | +8.3 | |
| Undecided | 56.7 [48.2, 59.9] | 64.5 [59.3, 65.3] | +7.8 |
| Benchmark | Midpoint ROC-AUC range |
|---|---|
| SST-2 | 0.956–0.975 |
| Rotten Tomatoes | 0.925–0.956 |
| TweetEval sentiment | 0.953–0.980 |
| Reddit Depression | 0.978–0.991 |
| Hyperpartisan | 0.604–0.702 |
| Cross-cultural Reddit | 0.615–0.687 |
| Model | Concept | Reference | Addition | ||
|---|---|---|---|---|---|
| Gemma 2 2B | Sentiment | Descr. | +0.3034/-0.3034 | +0.0008/-0.0015 | +0.0543/+0.0549 |
| Gemma 2 2B | Sentiment | Attit. | +0.2873/-0.2873 | +0.0005/-0.0009 | +0.0399/+0.0403 |
| Gemma 2 2B | Pol. Stance | Descr. | +0.4435/-0.4435 | +0.0062/-0.0033 | +0.0150/+0.0187 |
| Gemma 2 2B | Pol. Stance | Undecided | +0.4456/-0.4456 | +0.0062/-0.0033 | +0.0151/+0.0189 |
| Gemma 2 2B | Depression | Ordinary | +0.2220/-0.2220 | +0.0028/+0.0016 | +0.0521/+0.0517 |
| Gemma 2 2B | Depression | Absence | +0.2123/-0.2123 | +0.0024/+0.0016 | +0.0495/+0.0488 |
| Model | Concept | Reference | Addition | ||
|---|---|---|---|---|---|
| Gemma 2 2B | Sentiment | Descr. | +0.0556/-0.0556 | +0.0074/-0.0070 | +0.0157/-0.0067 |
| Gemma 2 2B | Sentiment | Attit. | +0.0374/-0.0374 | +0.0258/-0.0244 | +0.0225/-0.0089 |
| Gemma 2 2B | Pol. Stance | Descr. | +0.0409/-0.0409 | +0.0393/-0.0391 | +0.0192/-0.0172 |
| Gemma 2 2B | Pol. Stance | Undecided | +0.0844/-0.0844 | +0.0801/-0.0746 | +0.0452/-0.0425 |
| Gemma 2 2B | Depression | Ordinary | +0.0933/-0.0933 | +0.0068/-0.0017 | +0.0206/+0.0073 |
| Gemma 2 2B | Depression | Absence | +0.0265/-0.0265 | +0.0066/-0.0041 | +0.0093/+0.0008 |
| Mean margin change | Both margins improve | |||
| Action | Positive | Opposite | Conditions | Held evals. |
| Bipolar: positive/opposite | ||||
| Mean-line amplification | ||||
| amplification | ||||
| amplification | ||||
| Unipolar: positive/reference | ||||
| Larger minimum change | Both margins improve | |||
|---|---|---|---|---|
| Source geometry | Mean line | Mean line | ||
| ( ) | ||||
| ( ) | ||||
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing |
|---|---|---|---|---|---|---|
| Full conceptor | 58.0 [49.5, 66.5] | 43.0 [34.5, 52.0] | 43.5 [34.5, 52.5] | 25.5 [18.0, 33.5] | 45.9 [37.8, 54.1] | 65.8 [57.1, 74.0] |
| Rank-one conceptor | 56.0 [47.0, 65.0] | 46.0 [37.5, 54.5] | 40.0 [31.5, 48.5] | 26.5 [19.5, 34.0] | 49.5 [41.8, 57.1] | 62.2 [53.6, 70.4] |
| Mean-difference add. | 15.0 [8.5, 22.0] | 6.0 [2.0, 11.0] | 12.0 [6.5, 18.0] | 5.5 [2.0, 9.5] | 8.2 [4.1, 13.3] | 4.6 [1.0, 9.2] |
| Pooled-mean add. | 13.0 [7.5, 19.5] | 8.5 [4.0, 14.0] | 10.5 [5.5, 16.5] | 5.0 [1.5, 9.0] | 8.2 [4.1, 13.3] | 9.7 [4.6, 15.8] |
| Mean-line amp. | 8.5 [4.0, 14.0] | 5.0 [1.5, 9.5] | 11.5 [6.0, 17.5] | 5.0 [1.5, 9.0] | 7.7 [3.6, 12.2] | 9.2 [4.1, 14.8] |
| Eight-shot ICL | 78.0 [71.0, 84.5] | 89.5 [84.5, 94.0] | 63.0 [54.0, 71.5] | 73.0 [65.0, 81.0] | 94.9 [90.8, 98.5] | 87.8 [82.1, 92.3] |
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing |
|---|---|---|---|---|---|---|
| Full conceptor | 46.0 [37.5, 54.5] | 43.0 [35.0, 51.5] | 24.5 [17.0, 32.5] | 21.5 [14.5, 29.0] | 58.2 [50.5, 65.8] | 53.1 [43.9, 62.2] |
| Rank-one conceptor | 58.0 [49.5, 66.5] | 50.0 [41.5, 58.5] | 31.5 [23.5, 40.0] | 28.0 [20.5, 36.0] | 57.1 [49.0, 65.3] | 56.1 [47.4, 64.8] |
| Mean-difference add. | 7.5 [3.0, 13.0] | 6.0 [2.0, 10.5] | 5.5 [1.5, 10.0] | 7.0 [2.5, 12.0] | 10.7 [5.6, 16.3] | 6.6 [2.0, 11.7] |
| Pooled-mean add. | 4.5 [1.5, 8.5] | 8.5 [4.0, 13.5] | 5.5 [1.5, 10.0] | 10.0 [5.0, 16.0] | 9.7 [4.6, 15.3] | 8.2 [3.6, 13.3] |
| Mean-line amp. | 3.0 [0.5, 6.5] | 8.0 [3.5, 13.5] | 2.0 [0.0, 5.0] | 3.0 [0.5, 6.5] | 8.2 [4.1, 12.8] | 9.2 [4.6, 14.8] |
| Eight-shot ICL | 81.0 [74.5, 87.0] | 87.0 [81.5, 92.0] | 55.5 [46.5, 64.5] | 61.0 [52.0, 70.0] | 93.4 [88.8, 96.9] | 90.3 [85.7, 94.4] |
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing |
|---|---|---|---|---|---|---|
| Full conceptor | 64.0 [56.5, 71.5] | 56.0 [48.0, 64.5] | 44.0 [35.5, 53.0] | 34.5 [27.0, 42.0] | 33.2 [25.5, 40.8] | 57.7 [50.5, 64.3] |
| Rank-one conceptor | 64.5 [57.0, 72.0] | 64.0 [56.0, 71.5] | 38.5 [30.0, 47.0] | 31.5 [24.0, 39.0] | 45.4 [37.8, 53.6] | 68.9 [62.2, 75.0] |
| Mean-difference add. | 20.5 [14.0, 27.5] | 15.0 [9.5, 21.0] | 18.0 [11.5, 25.0] | 10.0 [5.0, 15.5] | 5.6 [2.0, 9.7] | 19.9 [13.3, 27.0] |
| Pooled-mean add. | 22.0 [15.0, 29.5] | 17.5 [11.5, 24.0] | 25.0 [17.5, 32.5] | 11.5 [6.5, 17.0] | 3.6 [1.0, 6.6] | 18.9 [12.2, 26.0] |
| Mean-line amp. | 19.5 [13.0, 26.5] | 14.0 [8.5, 20.0] | 22.0 [15.0, 29.5] | 18.5 [12.0, 25.5] | 6.1 [2.6, 10.7] | 19.9 [12.8, 27.6] |
| Eight-shot ICL | 80.0 [74.0, 86.0] | 88.0 [83.0, 93.0] | 42.0 [33.5, 50.5] | 68.0 [60.0, 76.0] | 94.9 [90.8, 98.0] | 82.7 [77.6, 87.8] |
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing |
|---|---|---|---|---|---|---|
| Full conceptor | 61.5 [53.5, 69.5] | 58.0 [49.0, 67.0] | 54.0 [45.0, 63.0] | 49.5 [40.5, 58.5] | 56.6 [48.0, 64.8] | 60.7 [53.6, 67.9] |
| Rank-one conceptor | 62.0 [54.0, 70.0] | 60.5 [52.0, 69.0] | 48.5 [39.5, 57.5] | 49.5 [41.0, 58.0] | 59.2 [51.0, 67.3] | 67.9 [61.7, 74.0] |
| Mean-difference add. | 21.0 [14.0, 28.5] | 25.5 [18.0, 33.5] | 27.0 [19.5, 35.0] | 20.5 [14.0, 27.5] | 34.2 [26.0, 42.3] | 23.0 [15.8, 30.6] |
| Pooled-mean add. | 22.5 [15.5, 30.0] | 33.0 [25.0, 41.5] | 27.5 [19.5, 36.0] | 23.0 [15.5, 31.0] | 30.6 [23.0, 38.8] | 28.6 [20.9, 36.7] |
| Mean-line amp. | 18.5 [11.5, 26.0] | 20.0 [13.0, 27.5] | 24.5 [16.5, 33.0] | 12.0 [7.0, 18.0] | 21.4 [14.3, 29.1] | 25.5 [18.4, 33.2] |
| Eight-shot ICL | 75.5 [68.5, 82.0] | 75.5 [68.0, 83.0] | 21.0 [14.0, 28.5] | 57.5 [48.5, 66.5] | 75.0 [67.3, 82.1] | 70.4 [64.3, 76.5] |
| Method | Sent.+ | Sent. | Left | Right | Dep. | Wellbeing |
|---|---|---|---|---|---|---|
| Full conceptor | 48.0 [39.5, 57.0] | 36.0 [27.5, 44.5] | 35.0 [26.5, 43.5] | 31.5 [24.0, 39.5] | 46.9 [39.3, 54.6] | 56.6 [48.0, 65.3] |
| Rank-one conceptor | 51.5 [43.0, 60.0] | 42.0 [33.5, 50.5] | 48.0 [39.5, 56.5] | 38.7 [30.0, 47.7] | 49.0 [40.8, 57.1] | 52.0 [43.4, 60.7] |
| Mean-difference add. | 17.5 [10.5, 25.0] | 12.0 [6.5, 18.0] | 6.0 [2.0, 10.5] | 10.5 [5.5, 16.5] | 9.7 [5.6, 14.8] | 12.2 [6.6, 18.4] |
| Pooled-mean add. | 13.0 [7.0, 19.5] | 11.5 [6.0, 17.5] | 8.5 [4.0, 14.0] | 9.0 [4.5, 14.0] | 12.2 [7.1, 18.4] | 12.8 [7.1, 18.9] |
| Mean-line amp. | 5.5 [2.0, 10.0] | 8.0 [3.0, 14.0] | 7.5 [3.0, 12.5] | 5.5 [2.5, 9.0] | 11.2 [6.6, 16.3] | 13.8 [8.2, 20.4] |
| Eight-shot ICL | 84.5 [78.0, 90.0] | 93.5 [89.5, 97.0] | 47.5 [38.5, 56.5] | 58.0 [49.0, 67.0] | 98.5 [96.4, 100.0] | 87.2 [81.1, 92.9] |
| Input | BASE | Full conceptor filter |
|---|---|---|
| Positive sentiment So flavorful and has just the perfect amount of heat. | […] this dish really knows how to balance out all the different flavors without overpowering them […] | […] every bite is like a party on my taste buds, but it’s not overwhelming at all, which makes me love it even more. […] |
| Negative sentiment Again, no plot at all. | […] every scene is just another random event that doesn’t seem connected to anything else. […] | […] every scene is just another disconnected moment that doesn’t seem to be building towards anything meaningful. […] |
| Left-leaning expression I blame deregulation for the accident. | […] we can’t just point fingers at one thing - there are many factors that led to this disaster […] | […] it’s absolutely appalling how lax regulations have become over the years, allowing companies to prioritize profits over people’s safety. […] |
| Right-leaning expression I believe in the future of crypto. | […] there are so many exciting developments happening within the space right now […] | […] financial freedom, especially considering its decentralized nature which allows individuals to have more control over their own money without relying on traditional institutions […] |
| Depressive expression I can’t hold back the tears anymore. | […] holding all these emotions in is slowly suffocating me, and I just need someone to understand why I’m breaking down now. | […] it feels overwhelming as my heart breaks down into uncontrollable sobs, releasing this weight of sadness and grief that I thought I could carry alone no more. |
| Affirmative wellbeing My mood is light and warm. | […] it’s amazing how something as simple as nature can lift your spirits and put you at ease. […] | […] The warmth inside me radiates outward, filling every moment with an infectious sense of joy that makes everyone I meet smile along with me. […] |
| Sentiment | Political leaning | |||||
|---|---|---|---|---|---|---|
| Model | Poles | Descr. | Attitud. | Poles | Descr. | Undecided |
| Gemma-2-2B | 35.7 | 9.9 | 11.5 | 18.4 | 6.5 | 6.1 |
| Gemma-2-9B | 29.5 | 8.8 | 10.2 | 19.8 | 6.1 | 6.8 |
| Qwen-2.5-3B | 37.9 | 18.7 | 22.7 | 24.9 | 14.8 | 16.5 |
| Llama-3.1-8B | 32.0 | 14.8 | 15.9 | 20.2 | 9.3 | 9.8 |
| OLMo-3-7B | 11.9 | 6.0 | 5.6 | 11.1 | 3.1 | 3.9 |
Representation Geometry in Language Models
Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.
See all 446 papers