Concept Representations in Vector Space: Quantifying Sharedness and Idiosyncrasy of Meaning
Authors: Simon Josef Durstewitz, Denis Buehler
Organizations: Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark · Institut Jean Nicod, École normale supérieure – PSL, Paris, France
Concepts are the basic units through which we navigate the world and communicate meaning. But do you and I really mean the same thing when we think or speak about a concept? The present study develops a measurement framework for modeling and analyzing interindividual variation in concept representations through sentence embeddings, dense vector representations mapping entire utterances into a shared geometric space. The framework is developed in a theoretical argument and validated, demonstrated, and subjected to robustness checks in a pre-registered study. Short personal definitions, elicited from N=125 participants, are encoded as sentence embeddings and submitted pairwise to a crossed random effects model yielding two concept-level quantities: sharedness (Sb), the expected dissimilarity between two randomly drawn individuals, and idiosyncrasy (Ib), the person-specific share of total dyadic variance. Two validity tests support the approach: the embedding space fully recovers concept and domain organization from participant definitions even with the target word excluded, and a parameter-free classifier achieves 0.630 accuracy in a within-participant extensional prediction task against a chance baseline of 0.500 and a consistency ceiling of 0.893. The framework finds theoretically interpretable variation in Sb and establishes Ib as a reliable source of dyadic variance across concepts. Results replicate across three encoders varying in architecture and capacity, providing a reproducible and scalable instrument for the empirical study of meaning variation.
Figures & tables
Figure 1: The center column shows the main analysis, the left column reports validity assessments, and the right column robustness checks using alternative (Alt) encoders.
Figure 2: The illustration comprises four levels, from top to bottom: outside the mind, mind, linguistic access, and elicitation strategy. A target word is taken to express a concept and representations are then elicited in linguistic format to empirically open these up for comparison.
Figure 3: Arranging the concept set along two dimensions affords a comparatively rich set of analytical comparisons from a small number of concepts. Within-domain and second-order comparisons are also less susceptible to possible non-uniform semantic resolution (see also Section 6.1 ). Note, however, that no comparison supports causal attribution, as each cell contains only one concept, and concepts may differ on more than the controlled dimensions. The purpose of the demonstration is to surface and explore variation.
Figure 4: Each cell contains the mean cosine similarity across all participant embedding pairs combining concept b with concept b′ . Diagonal cells reflect mean within-concept similarity across participants; off-diagonal cells reflect mean cross-concept similarity.
Option A
Option B
Concept
Item
N
Predicted
Chosen
Predicted
Chosen
Accuracy
Plant
Item 1
116
31
41
85
75
0.603
Item 2
116
113
95
3
21
0.828
Flower
Item 1
118
79
63
39
55
0.508
Item 2
118
25
44
93
74
0.636
Rose
Item 1
117
26
62
91
55
0.538
Table 1: Item-level predictive validity under the primary encoder. Predicted and chosen counts are given for each response option. N is the number of participants with a valid definition and a valid forced-choice response for that item. Chance baseline =0.500 .
Figure 5: Axis limits share a common half-width determined by the maximum data extent across triplets, enabling comparison of cloud size and concentration across panels. Each row is centered on its respective triplet; absolute positions across rows are therefore not comparable. Color intensity represents kernel density normalized to the global maximum, with warmer colors indicating higher density. HDR contours delineate the 25, 50, 75, and 90% highest-density regions for each concept.
Figure 6: Each cell displays the point estimate and 95% bias-corrected bootstrap CI for Sb and Ib for a given concept.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
Table 8
Domain
Concept
Item
Descriptions
Natural
Plant
1
(A) A tall oak tree rising from the ground in a wild forest.
(B) A small ornamental orchid kept indoors in a pot.
2
(A) A dense green bush with broad leaves.
(B) A blooming bush covered in red, orange, and pink dahlias.
Flower
1
(A) A carefully cultivated tulip growing in a garden.
(B) A wild dandelion growing in a field.
Appendix
Table A3: Forced-choice typicality items. The task consisted of 18 items, two per concept. For each item, participants selected the description that most closely aligned with their own understanding of the concept. Each pair of descriptions contrasts two options differing along dimensions relevant to that concept.
Concept
Definition (verbatim)
Exclusion criterion
Blazer
A blazer is I was really not sure what it was when I was answering cause my english was not englishing please dont take this concept into account :)
Explicit request to not be included
Plant
A plant is boring. If it’s very green it can be pretty but I can’t water them. I always forget. And that’s it.
Largely affective associations
Rose
A rose is pink flower, with soft petals. “C’est une folie de haïr toutes les roses parce qu’une épine vous a piqué”
Non-English content
Clothing
Clothing is something you wear.
Minimal definitional content; character minimum circumvented because of drop-out
Board Game
A board game is an event where you can really easily be mad at your friends.
Minimal definitional content
Chess
Chess is a very funny game. Especially in competitions. I love the book “the chess player” by Zweig which tells the story of a schinzophren man.
No definitional content about the target concept
Appendix
Table A4: Definitions excluded prior to analysis, reproduced verbatim. Spelling errors present in the original responses are retained.
An increasing number of disciplines recognize the explanatory potential of meaning-bearing mental structures through which individuals interpret the world and guide their behavior. Concept representations constitute a core class of these structures, emerging from individuals' experiences with concept referents and encoding and organizing their understanding of them. Recognizing a private level of meaning, where concepts are not objectively defined by rules in the world but constitute subjective structures in individuals' minds, raises a fundamental question: do different individuals actually mean the same thing when they invoke the same concept? As latent structures, concept representations are difficult to access and model empirically. The present study introduces a measurement framework that makes such representations empirically accessible through textual traces elicited under standardized conditions. Using transformer-based sentence embeddings, the framework maps standardized responses elicited for concepts into a high-dimensional shared vector space, opening concept representations to the systematic analysis of interindividual variation. It develops three analytical avenues: (1) comparing representational sharedness across concepts, (2) identifying variables that correlate with representational variation, and (3) examining how focal actors or subpopulations deviate from a reference population. Each avenue is illustrated through an empirical proof-of-concept drawing on a large-scale free-association dataset. Together, the demonstrations show how researchers from across disciplines can move from a research question and data source to analysis and interpretation. An accompanying GitHub repository provides a ready-to-use implementation of the framework for existing and custom datasets.
Simon Josef Durstewitz
Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark
The advent of distributional semantics as a framework for computational semantics marked a paradigm shift in the nature of representations, by which meanings are represented as dense vectors commonly known as word embeddings. Computational approaches to historical or typological language comparison, on the other hand, still mostly rely on discrete representations, while the potential of embedding-based methods remains largely untapped in this context. Recent studies have proposed concept embeddings as a language-independent, onomasiological counterpart to word embeddings, based on networks of cross-linguistic colexification patterns. While the resulting concept embeddings reflect meaningful semantic properties and relations, they suffer from a lack of scalability, since they rely on transductive embedding approaches which only learn embeddings for nodes present in the training data. In this study, we present an inductive framework for learning concept embeddings, combining the structural properties of full and partial colexification networks with semantic information from short concept descriptions. This framework allows the embedding of unseen concepts based on a short definition. We show that the resulting concept embeddings correlate with various cross-linguistic measures of similarity or relatedness, while generalizing well to new concepts that were absent from the training data.
Arne Rubehn
Chair for Multilingual Computational Linguistics, University of Passau, Passau, Germany
Understanding how neural models represent human-interpretable concepts is challenging. Prior work has explored linear concept subspaces from diverse perspectives, such as probing and concept erasure. We introduce a unified framework to study these subspaces along two axes: containment, which tests if a concept is fully represented in a subspace but not outside it, and disentanglement, which tests for isolation from other concepts. In experiments on both text and speech models, we first highlight that concept subspaces may not be uniquely determined, and discuss the implications for concept subspace analysis. Then, we compare properties of concept subspaces estimated using five estimators, proposed in different communities. We find that (1) the choice of estimator impacts the containment and disentanglement properties; (2) the state-of-the-art concept erasure method, LEACE, performs well on both testing axes, but still struggles to generalize to unseen data; and (3) in HuBERT speech representations, phone information is both contained and disentangled from speaker information, while speaker information is hard to contain in a compact subspace, despite being disentangled from phones.
Burin Naowarat, Hao Tang, Sharon Goldwater
The Centre for Speech Technology Research School of Informatics, University of Edinburgh, United Kingdom