TOPol: Reconstructing and Interpreting Multidimensional Semantic Polarity Fields
Organizations: University of Geneva · Bern Unversity of Applied Sciences · University of Twente · LGT Bank
Abstract
Polarity in natural language processing has long been operationalized as a single axis: the sentiment construct. We challenge this one-dimensional view by asking what a topic's documents are bipolar about, without specifying an answer in advance. TOPol inverts the usual concept-direction paradigm: rather than mapping a named construct to one direction, it maps a topic to the full set of statistically reproducible bipolar semantic axes its documents support in a transformer's residual stream, and asks only afterwards which of them correspond to a known construct. These axes need not be evaluative. The method is a recursive, orthogonal-by-construction bipartition of each topic's document cloud, with a candidate axis promoted to a finding only if it survives one pre-specified split-half stability gate. We calibrate against a known answer first: on 9,986 Amazon reviews (14 discovered topics) in Gemma 3 4B, the star-rating difference-in-means vector serves as a ruler known by construction to be sentiment. Discovery blindly re-derives the sentiment axis in every topic (cosine 0.71-0.97) without ever seeing the rating label, while the same gate certifies 543 axes, of which 106 are near-orthogonal to sentiment yet equally reproducible; each topic supports 33-45 mutually orthogonal axes, and sentiment axes resolve in later layers than the rest. Differential vocabulary, an internals-grounded natural-language decoder and vocabulary-based clustering agree on what the non-sentiment axes contrast; clustering by shared vocabulary surfaces 99 families, 9 recurring across topics. Direction similarity to the ruler predicts an independent sentiment estimate only weakly (). Polarity is better treated as a property of a topic's representational geometry, with a measurable rank and a per-axis depth, than as one universal scalar.
Figures & tables
| # | Topic label | Docs | Candidates Passed | Identification | Found at depth | Excess axes |
|---|---|---|---|---|---|---|
| 0 | Music/CDs | 1285 | 53 45 | 0.812 | 4 | 8 |
| 1 | Defective/returned merch. | 1095 | 49 35 | 0.813 | 4 | 4 |
| 2 | How-to/reference books | 1077 | 48 36 | 0.929 | 4 | 9 |
| 3 | Movies/TV (DVD) | 1049 | 47 40 | 0.973 | 2 | 8 |
| 4 | Home/tools/kitchen | 831 | 57 42 | 0.953 | 2 | 7 |
| 5 | Fiction (critical) | 830 | 51 43 | 0.854 | 2 | 14 |
| Topic# | Top terms, pole A | Top terms, pole B | ||||
|---|---|---|---|---|---|---|
| 7 | 10 | 1.45 | 0.29 | church, history, jesus, russian, christian, science, bible, catholic | things, don, written, didn, reviews, reviewer, thoughtful | |
| 7 | 20 | 0.47 | 0.33 | bible, jesus, god, bush, moore, truth, history | good, life, human, culture, government, writing, parents | |
| 6 | 34 | 0.45 | 0.14 | software, support, xp, windows, computer, router, usb | sound, quality, camera, price, headphones, speakers, lens | |
| 2 | 4 | 0.84 | 0.03 | don, waste, workout, dvd, money, waste money | excellent, information, comprehensive, wonderful, guide, helpful |
| Topic | |||
|---|---|---|---|
| 3 Movies/TV | 40 | ||
| 11 Food/care/pet | 33 | ||
| 7 Religion/politics | 36 | ||
| 4 Home/tools | 42 | ||
| 2 How-to/reference | 36 | ||
| 5 Fiction (critical) | 43 |
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
| Family | # topics | Topics |
| 84 (universal) | 14 | all |
| 80 | 8 | 0,2,3,5,8,9,10,12 |
| 78 | 6 | 0,3,4,5,11,13 |
| 64 | 5 | 0,1,6,8,12 |
| 63 | 4 | 0,2,5,7 |
| 12 | 3 | 0,3,8 |
| Fam. | Axes | Topics | Pole A terms / Pole B terms |
|---|---|---|---|
| 73 | 33 | 5,7,8 | government, history, american / book, don, good |
| 30 | 24 | 0,1,2,4,6,7,10,11 | like, questions, didn / love, thing, guide |
| 72 | 18 | 5,7,8 | don, isn, ve / history, maybe, time |
| 42 | 16 | 0,2,3,4,11,13 | bad, worst, like / great, cd, love |
| 60 | 12 | 5,8 | num pages, idea, like book / romance, author, mystery |
| 31 | 4 | 4,6 | belt, item, unit / love, just, lot |
LLM Interpretability
Large language model (LLM) interpretability studies how internal representations and computations produce outputs, and how they can be explained or controlled.
See all 829 papers