Probing the Geometry of Color Representations in Multilingual Language Models

Pablo Loyola; Edison Marrese-Taylor; Andrés Hoyos-Idrobo

cs.CLOct 13, 2026

Probing the Geometry of Color Representations in Multilingual Language Models

Authors: Pablo Loyola, Edison Marrese-Taylor, Andrés Hoyos-Idrobo

Organizations: Rakuten Group, Inc. · Graduate School of Engineering, The University of Tokyo, Japan

Abstract

Do multilingual language models learn a universal internal geometry for color, and is it perceptually grounded or primarily linguistic? We decompose color representations into two geometric axes: (P1) alignment to perceptual color space, and (P2) cross-lingual symmetry. Analyzing layerwise embeddings across models and multilingual datasets, we find that perceptual grounding is weak: alignment to CIE-LAB peaks in intermediate layers, then collapses toward output layers. Cross-lingual symmetry degrades with depth, suggesting later layers encode language-specific structure. Functionally, we show that geometric properties predict task performance: models achieve 96.7% accuracy on color classification at layers where P1 alignment peaks, with P1 alignment (r=0.70) significantly correlating with downstream capabilities. Our results reveal a dissociation: text-only LMs learn universal color-language geometry, not universal color-perception geometry, providing diagnostics for where perceptual structure is preserved in language models.

Figures & tables

Panel of Figure 1 — Figure 1: Analysis pipeline. We extract layer-wise representations $\mathbf{Z}^{(l)}$ from language models processing color descriptions $(\mathbf{c}_{i},\mathbf{d}_{i},\ell_{i})$ from three datasets (MLMC, M-ColorNames, Munsell) spanning multiple languages; one representative example per dataset is shown (colour value, description, language). Two geometric axes are computed independently from $\mathbf{Z}^{(l)}$ via pairwise, sample-wise kernels (an $N\times N$ object built from $\mathbf{Z}^{(l)}$ , not a feature–feature one) and quantify (P1) perceptual alignment, via kernel alignment (CKA) between the model kernel $\mathbf{K}_{\mathcal{M}}^{(l)}$ and a CIE–LAB-based perceptual kernel, and (P2) cross-lingual symmetry, via congruence between matched language-pair kernels. A spectral diagnostic, effective rank $\mathrm{ER}^{(l)}$ , is likewise computed directly from $\mathbf{Z}^{(l)}$ and correlates with both axes (shown alongside Representations) without being derived from either, providing a unifying explanation for when geometric quality degrades. Functional validation via linear probes on six color tasks shows both axes carry downstream predictive value, supporting the paper’s central finding: color geometry in multilingual LMs is partly universal across languages but not strongly perceptually grounded .

Table 2

MLMC Dataset (pragmatic)
ZH	螢光藍 ( fluorescent blue )	RU	желто зеленый (yellow-green)
	亮紅色 ( bright red )		ультрамариновый (ultramarine)
KO	연한파랑 ( light blue )	PL	niebieski jasny (light blue)
	형광 핑크 ( fluorescent pink )		różowoczerwony (pink-red)
ES	verde oscuro (dark green)	SV	turkås (turquoise)
	morada (purple)		lilla (purple)

Table 3: Sample color descriptions from both datasets. Top : MLMC (pragmatic, 2 samples

\times

10 languages). Bottom : M-ColorNames (creative, 2 samples

\times

6 languages). Swatches show target colours; descriptions in italics with English translations in parentheses (omitted for English entries).

Figure 2: Perceptual alignment peaks early, collapses late. CKA between model representations and CIE-LAB at each layer. Filled circles mark peak per model. MLMC-EN and M-ColorNames show early-to-mid “perceptual window” with reduced alignment toward final layers. Munsell exhibits higher, more persistent alignment. Uncertainty estimates in Appendix C .

Model	Peak KA ( $\mu\pm\sigma$ )	Final KA ( $\mu\pm\sigma$ )	Decay ( $\mu\pm\sigma$ )	Peak layer ( $\mu\pm\sigma$ )
Gemma-2-2B	0.257 $\pm$ 0.025	0.091 $\pm$ 0.006	2.84 $\pm$ 0.28	1.0 $\pm$ 0.0
Gemma-2-9B	0.254 $\pm$ 0.028	0.169 $\pm$ 0.012	1.50 $\pm$ 0.10	4.4 $\pm$ 4.2
Llama-3.1-8B	0.240 $\pm$ 0.016	0.140 $\pm$ 0.021	1.73 $\pm$ 0.17	0.0 $\pm$ 0.0
Mistral-7B	0.242 $\pm$ 0.018	0.103 $\pm$ 0.015	2.36 $\pm$ 0.23	0.0 $\pm$ 0.0
Qwen2.5-7B	0.202 $\pm$ 0.025	0.079 $\pm$ 0.013	2.59 $\pm$ 0.30	0.0 $\pm$ 0.0

Table 4: Matched-

n

perceptual control (P1) . MLMC-EN subsamples matched to Munsell size (

n=122

), 5 seeds. KA is normalized centered CKA to LAB. Matching largely reduces the apparent Munsell advantage while preserving the peak-to-final decay profile across model families.

Model	Subset	$n$	Peak KA	Final KA	Peak layer
Gemma-2-2B	BCT	27	0.4079	0.0934	1
Gemma-2-2B	non-BCT	27	0.0976	0.0276	3
Gemma-2-9B	BCT	2 916	0.4033	0.1603	1
Gemma-2-9B	non-BCT	1 761	0.0970	0.0585	6
Llama-3.1-8B	BCT	2 916	0.3555	0.1340	0
Llama-3.1-8B	non-BCT	1 761	0.0901	0.0538	0

Table 5: BCT-conditioned analysis on MLMC-EN . Descriptions containing an explicit basic color term (BCT) exhibit higher perceptual alignment (peak and final CKA) across model families, linking lexical explicitness to more perceptually structured geometry. (Gemma-2-2B rows use a smaller evaluated subset.)

Figure 3: Cross-lingual congruence degrades with depth. Mean pairwise CKA between per-language kernels on matched color sets. Thick dashed line: model average; gray band: 95% model-bootstrap CI (robustness to model choice). High early congruence declines in later layers, with model-dependent dynamics.

Model	$L$	Congruence			Drift
Model	$L$	$l=0$	$l=\lceil L/2\rceil$	$l=L$	$l=L$
Gemma-2B	26	0.599	0.437	0.302	1.337
Gemma-9B	42	0.622	0.127	0.467	1.271
PaliGemma2-3B	26	0.557	0.541	0.338	1.296
Llama-8B	32	0.710	0.568	0.558	1.032
Mistral-7B	32	0.663	0.503	0.479	1.085

Table 6: Cross-lingual congruence across layers (matched-set kernels).

Dataset	Gemma-2B	Gemma-9B	PaliGemma2-3B	Llama-8B	Mistral-7B	Qwen-7B
MLMC	0.876	0.930	0.889	0.781	0.941	0.986
M-ColorNames	0.820	0.929	0.907	0.761	0.755	0.859
Munsell	0.942	0.980	0.935	0.851	0.943	0.996

Table 7: Correlation between effective rank and key P1 & P2 metrics (reported in the analysis).

Figure 4: Color task definitions. Left: Hue circle showing the warm/cool classification task. Warm colors (red-yellow, hue angles 0–90° and 330–360° with

C^{*}>20

) are highlighted in red; cool colors (yellow-green-blue-purple, 90–270°) are highlighted in blue. This task directly tests circular hue structure. Right: Summary of all six color tasks used in functional validation. Tasks probe different aspects of color space: LAB-based similarity (Task 1), hue discrimination (Task 2), lightness (Task 3), chroma (Task 4), hue regions (Task 5), and combined attributes (Task 6). All models achieved peak performance at early-to-mid layers (

l\in[2,\ldots,6]

), with degradation toward final layers, mirroring the P1 alignment profile.

Panel of Figure 5 — Figure 5: Functional validation of perceptual alignment (P1). Left: Layer-wise warm/cool classification accuracy for three models on MLMC (8 521 samples). Right: Correlation between P1 perceptual alignment (CKA) and warm/cool task accuracy across layers for Llama-3.1-8B on M-ColorNames (21 364 samples). Each point represents one layer, colored by depth. Linear fit shows significant positive correlation ( $r=0.70$ , $p=0.035$ ), demonstrating that P1 alignment predicts functional competence: layers with stronger perceptual grounding excel at color attribute tasks. The dashed red line indicates chance performance (50%).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Pooling	Peak KA	Peak layer	Final KA	Decay
Gemma-2-2B	max	0.2025	1	0.0500	4.05 $\times$
	mean	0.2041	1	0.0422	4.84 $\times$
	last	0.0301	1	0.0121	2.50 $\times$
Gemma-2-9B	max	0.2014	2	0.0946	2.13 $\times$
	mean	0.2154	1	0.0950	2.27 $\times$
	last	0.0784	1	0.0392	2.00 $\times$

Appendix

Table 8: Pooling robustness (P1) . Mean and max pooling give broadly consistent peak and decay across families, while last-token pooling is unstable (collapsing alignment for Gemma/Mistral but yielding higher peaks for Llama/Qwen). We use mean/max pooling as the default and treat last-token as an ablation.

Representation Geometry in Language Models

Representation geometry in language models studies how embeddings and hidden states organize concepts, computation, and behavior.

See all 417 papers

Beyond Color Geometry: Evaluating Human-Like Color Representations in Vision Models
Jul 15, 2026Ayan Igali, Pakizar ShamoiVisual Representation Learning Fine-Grained Visual Perception
School of Information Technology and Engineering Kazakh–British Technical University Almaty, Kazakhstan
Human-AI Perceptual Alignment by Playing Hues and Cues
Aug 7, 2026Nuria Alabau-Bosque, Jorge Vila-Tomás, Paula Daudén-Oliver +3VLM Evaluation Vision-Language Grounding
Image Processing Lab, Universitat de València, Carrer del Catedrátic José Beltrán Martinez, Paterna, 46980, Spain
Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere
Oct 18, 2026Mikaela Martins, Jonathan DunnMultilingual Language Models Concept Localization in Language Models
Department of Linguistics, University of Illinois Urbana-Champaign

Jul 15, 2026J/K move · Enter open · S save

Beyond Color Geometry: Evaluating Human-Like Color Representations in Vision Models

Ayan Igali, Pakizar Shamoi

School of Information Technology and Engineering Kazakh–British Technical University Almaty, Kazakhstan

Visual Representation Learning Fine-Grained Visual Perception Human Visual Perception

Open paper AskPDF arXiv

Do vision models see colors the way humans do? Existing evaluations of color representations usually compare them with geometric spaces such as CIELAB or with discrete color labels. These references capture perceptual distance or category membership, but not the graded way in which people organize colors. We evaluate color grounding against a fuzzy perceptual model with 86 graded categories fitted to human survey data. The framework can be applied to any image encoder and measures three complementary properties: category boundaries, category compactness, and graded alignment beyond what color geometry alone can explain. Across eleven Vision Transformer encoders, the category-level results are broadly similar, whereas graded alignment differs substantially. Masked Autoencoders achieve the strongest beyond-geometry alignment, with confidence intervals that do not overlap those of the other encoders. A layer-wise analysis further shows that masked reconstruction preserves this structure toward the output. On natural images, MAE represents surface color globally, while language-supervised models encode color more strongly in relation to the foreground object. These results show that human-like color grounding has several distinct aspects that should not be reduced to a single score.

Abstract

Representation Geometry in Language Models

Explore similar work

Beyond Color Geometry: Evaluating Human-Like Color Representations in Vision Models

Human-AI Perceptual Alignment by Playing Hues and Cues

Where Are Language-Specific Features Represented in Multilingual LLMs? Everywhere