cs.CLNov 14, 2026

The Geometry of Conceptors in Neural-Activation Spaces

Authors: Ilias Triantafyllopoulos, Young Min Cho, Ren Tao, Miranda Muqing Miao, Sunny Rai, Lyle Ungar, Sharath Chandra Guntuku, Sunghye Cho, 

Organizations: New York University · University of Pennsylvania · University of Cincinnati · Linguistic Data Consortium, University of Pennsylvania

Abstract

Conceptors represent concepts as graded collections of directions in neural-activation space rather than as single vectors. We develop a geometric account of how these representations depend on the contrast used to define a concept: opposing values in bipolar fitting, or presence and absence in unipolar fitting. We derive an exact decomposition of a centered pooled conceptor into an average pole filter, a mean-separation update, and a covariance-shape update. Experiments across five language models on sentiment, political leaning and stance, and depression expression show that both updates improve reconstruction of held-out contrasts. Additional directions preserve variation beyond the mean-difference line, and changing the comparator changes the benefit of these directions. Shape, size, and overlap each show up downstream: in amplification experiments, bipolar fits raise evidence for both poles more consistently than unipolar fits; conceptor size (the quota) correlates positively with layerwise probe performance; and cross-concept overlap quantifies interference under subspace removal, linking shared directions to Boolean composition. In an additional generation evaluation, conceptor filters achieve higher target-expression win rates against an unsteered reference than three mean-based controls. Together, these findings explain how fitting choices shape conceptor geometry and support bipolar fitting when the objective is to amplify both opposing poles.

Figures & tables

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Conceptors for Semantic Steering

    May 6, 2026Ilias Triantafyllopoulos, Young-Min Cho, Ren Tao +6Language Model SteeringRepresentation Steering

  2. Concept Subspaces Compute Beyond the Logit Lens: A Weights-Only Test for Locating Representations Upstream of Readout

    Sep 30, 2026Aojie Yuan, Zhiyuan Julian Su, Haiyue Zhang +1Transformer InterpretabilityRepresentation Geometry in Language Models