Concept Representations in Vector Space: Toward an Interdisciplinary Study of Meaning Variation

Simon Josef Durstewitz

cs.CLOct 18, 2026

Concept Representations in Vector Space: Toward an Interdisciplinary Study of Meaning Variation

Authors: Simon Josef Durstewitz

Organizations: Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark

Abstract

An increasing number of disciplines recognize the explanatory potential of meaning-bearing mental structures through which individuals interpret the world and guide their behavior. Concept representations constitute a core class of these structures, emerging from individuals' experiences with concept referents and encoding and organizing their understanding of them. Recognizing a private level of meaning, where concepts are not objectively defined by rules in the world but constitute subjective structures in individuals' minds, raises a fundamental question: do different individuals actually mean the same thing when they invoke the same concept? As latent structures, concept representations are difficult to access and model empirically. The present study introduces a measurement framework that makes such representations empirically accessible through textual traces elicited under standardized conditions. Using transformer-based sentence embeddings, the framework maps standardized responses elicited for concepts into a high-dimensional shared vector space, opening concept representations to the systematic analysis of interindividual variation. It develops three analytical avenues: (1) comparing representational sharedness across concepts, (2) identifying variables that correlate with representational variation, and (3) examining how focal actors or subpopulations deviate from a reference population. Each avenue is illustrated through an empirical proof-of-concept drawing on a large-scale free-association dataset. Together, the demonstrations show how researchers from across disciplines can move from a research question and data source to analysis and interpretation. An accompanying GitHub repository provides a ready-to-use implementation of the framework for existing and custom datasets.

Figures & tables

Figure 1: Free Association as One Elicitation Strategy. The demonstrations in the present study use free association from the Small World of Words dataset ( De et al., 2019a ) . A cue word is assumed to express a concept, eliciting associates that reflect content salient to the participants’ representation of that concept and are collectively assumed to reflect the underlying representation.

Figure 2: From Pairwise Distances to Three Analytical Avenues. Textual responses are (1) encoded as sentence embeddings $\mathbf{v}_{ib}$ , and (2) pairwise cosine dissimilarities $\delta_{ijb}$ form the point of departure for analyzing representational variation. Panel (3) illustrates three operations the dyadic statistical modeling enables. Panels (4) and (5) map each operation to its analytical avenue and exemplary demonstration.

Figure 3: Sharedness as a Function of Word Concreteness, Controlling for WordNet Synset Count. Each point represents the mean

S_{b}

of concepts falling within one of 100 equal-frequency bins along the residualized concreteness axis, used here for visual clarity. The red line shows the weighted regression of the residualized outcome on residualized concreteness, with 95% bootstrap confidence intervals derived from 2,000 resampled iterations.

Figure 4: Concept-Level Correlations Between Representational Differences and Predictor Distances. Panel (a) shows results for the 25 climate and ecology concepts analyzed against difference in absolute latitude; panel (b) shows results for the 25 sociopolitical and economic concepts analyzed against difference in age. Each point represents one concept. Filled points reach the significance threshold indicated by the asterisk markers.

r

= Pearson correlation between

\delta_{ij}

and

x_{ij}

, assessed via one-tailed Mantel test with 9,999 permutation iterations.

Figure 5: Pairwise Cosine Dissimilarity Distributions for Human–Human and LLM–Human Response Pairs. Each panel shows the kernel density estimate of pairwise cosine dissimilarities for human–human pairs (grey) and LLM–human pairs (blue). Vertical tick marks at the base of each panel indicate the mean pairwise dissimilarity for each distribution.

p

-values are from two-sided permutation tests (

9{,}999

iterations) assessing whether the mean LLM–human dissimilarity differs from the human–human baseline.

Oct 11, 2026

Concept Representations in Vector Space: Quantifying Sharedness and Idiosyncrasy of Meaning

Concepts are the basic units through which we navigate the world and communicate meaning. But do you and I really mean the same thing when we think or speak about a concept? The present study develops a measurement framework for modeling and analyzing interindividual variation in concept representations through sentence embeddings, dense vector representations mapping entire utterances into a shared geometric space. The framework is developed in a theoretical argument and validated, demonstrated, and subjected to robustness checks in a pre-registered study. Short personal definitions, elicited from

N = 125

participants, are encoded as sentence embeddings and submitted pairwise to a crossed random effects model yielding two concept-level quantities: sharedness (

S_b

), the expected dissimilarity between two randomly drawn individuals, and idiosyncrasy (

I_b

), the person-specific share of total dyadic variance. Two validity tests support the approach: the embedding space fully recovers concept and domain organization from participant definitions even with the target word excluded, and a parameter-free classifier achieves 0.630 accuracy in a within-participant extensional prediction task against a chance baseline of 0.500 and a consistency ceiling of 0.893. The framework finds theoretically interpretable variation in

S_b

and establishes

I_b

as a reliable source of dyadic variance across concepts. Results replicate across three encoders varying in architecture and capacity, providing a reproducible and scalable instrument for the empirical study of meaning variation.

Simon Josef Durstewitz, Denis Buehler

Copenhagen Center for Social Data Science (SODAS), University of Copenhagen, Denmark · Institut Jean Nicod, École normale supérieure – PSL, Paris, France

Topic pending

Ask PDF

May 2, 2026

A framework for analyzing concept representations in neural models

Understanding how neural models represent human-interpretable concepts is challenging. Prior work has explored linear concept subspaces from diverse perspectives, such as probing and concept erasure. We introduce a unified framework to study these subspaces along two axes: containment, which tests if a concept is fully represented in a subspace but not outside it, and disentanglement, which tests for isolation from other concepts. In experiments on both text and speech models, we first highlight that concept subspaces may not be uniquely determined, and discuss the implications for concept subspace analysis. Then, we compare properties of concept subspaces estimated using five estimators, proposed in different communities. We find that (1) the choice of estimator impacts the containment and disentanglement properties; (2) the state-of-the-art concept erasure method, LEACE, performs well on both testing axes, but still struggles to generalize to unseen data; and (3) in HuBERT speech representations, phone information is both contained and disentangled from speaker information, while speaker information is hard to contain in a compact subspace, despite being disentangled from phones.

Burin Naowarat, Hao Tang, Sharon Goldwater

The Centre for Speech Technology Research School of Informatics, University of Edinburgh, United Kingdom

Representation Disentanglement Neural Network Interpretability Concept Erasure

Ask PDF

Mar 10, 2026

The Prediction-Measurement Gap: Toward Meaning Representations as Scientific Instruments

Text embeddings have become central to computational social science and psychology, enabling scalable measurement of meaning and mixed-method inference. Yet most representation learning is optimized and evaluated for prediction and retrieval, yielding a prediction-measurement gap: representations that perform well as features may be poorly suited as scientific instruments. The paper argues that scientific meaning analysis motivates a distinct family of objectives - scientific usability - emphasizing geometric legibility, interpretability and traceability to linguistic evidence, robustness to non-semantic confounds, and compatibility with regression-style inference over semantic directions. Grounded in cognitive and neuro-psychological views of meaning, the paper assesses static word embeddings and contextual transformer representations against these requirements: static spaces remain attractive for transparent measurement, whereas contextual spaces offer richer semantics but entangle meaning with other signals and exhibit geometric and interpretability issues that complicate inference. The paper then outlines a course-setting agenda around (i) geometry-first design for gradients and abstraction, including hierarchy-aware spaces constrained by psychologically privileged levels; (ii) invertible post-hoc transformations that recondition embedding geometry and reduce nuisance influence; and (iii) meaning atlases and measurement-oriented evaluation protocols for reliable and traceable semantic inference. As the field debates the limits of scale-first progress, measurement-ready representations offer a principled new frontier.

Hubert Plisiecki

IDEAS Research Institute, Warsaw, Poland

Text Embeddings Text Embedding Evaluation Representation Geometry in Language Models Computational Social Science

Ask PDF

Abstract

Explore similar work

Concept Representations in Vector Space: Quantifying Sharedness and Idiosyncrasy of Meaning

A framework for analyzing concept representations in neural models

The Prediction-Measurement Gap: Toward Meaning Representations as Scientific Instruments