Learning Perceptual Geometry of Natural Odor Mixtures from LLM-Derived Descriptors

Farhad Pashakhanloo; Venkatesh N. Murthy

cs.AIOct 12, 2026

Learning Perceptual Geometry of Natural Odor Mixtures from LLM-Derived Descriptors

Authors: Farhad Pashakhanloo, Venkatesh N. Murthy

Organizations: Center for Brain Science, Harvard University · Department of Molecular and Cellular Biology, Harvard University · Kempner Institute for the Study of Natural and Artificial Intelligence, Harvard University

Abstract

The relationship between odor chemistry and perception remains incompletely understood. Many existing studies focus on the perception of single molecules, whereas natural odors contain tens to hundreds of components, adding extra layer of complexity to percept formation. Here, we explore the relationship between molecular composition and perceived similarity in natural odor objects. We study this using a database of more than 1,200 volatile-compound-in-food mixtures containing more than 8,000 molecular components, and use large language models (LLMs) as a scalable surrogate for deriving mixture-level perceptual descriptions. We find that language-derived representations provide useful supervision for predicting held-out mixture similarities from molecular composition. The prediction was consistently strongest when perceptual targets were derived from chemosensory prompts than from other sensory prompts. Further, during the mixture-level training, we learned low-rank molecular embeddings that enabled better prediction of held-out mixture similarities than component-overlap metrics, even for mixture pairs with no shared molecules. Although no single-molecule labels were used during training, the learned molecular geometry showed significant alignment with the independent human odor-descriptor space. Finally, we explored the use of external molecular features in our mixture-similarity prediction task. These features showed modest zero-shot alignment with mixture-level perceptual similarity. They improved prediction under limited supervision but offered little additional predictive benefit with the full training set. Together, these results show that language-derived descriptions can provide scalable supervision for predicting the perceptual organization of complex natural mixtures, contributing to a quantitative understanding of how molecular composition gives rise to odor perception.

Figures & tables

Figure 1: Natural-mixture dataset and language-derived representations. (A) Each mixture can be represented by a binary vector $x_{m}\in\{0,1\}^{P}$ specifying the presence or absence of different molecules. (B) Llama-3.1-8B-Instruct was prompted to describe each mixture using fixed adjective lists or free-form text, and corresponding perceptual targets were created from hidden state activations or word-embeddings. (C) Example prompt-responses of the model (see Table 1 for complete prompts). (D) Smell descriptor word cloud. The 45 most frequent descriptors pooled across five generation runs from the 10 adjectives pass; word size indicates frequency. (E–H) Two-dimensional t-SNE visualizations of mixture representations based on (E) binary molecular composition vectors, (F) pretrained word embeddings of 10 generated odor adjectives, (G) Llama activations from 10-adjective responses, and (H) Llama activations from free-form odor descriptions. All t-SNE projections were computed using pairwise cosine distances. Each point represents a mixture and is colored by food category. An interactive version of panels (F–H) is available online .

Figure 2: Predicting perceptual similarity from mixture compositions. (A) Schematics showing different kernels for comparing two mixtures. We show an example in which components 1 and 2 have similar embeddings, however, the mixture do not share those components. The identity and the diagonal kernels do not capture this effect leading to a lower similarity between mixtures. However, the kernel on the right can capture this effect due to the non-zero off-diagonal term $x_{a1}x_{b2}K_{12}$ . Note that high similarity of the components lead to a high value of $K_{12}=e_{1}^{\top}e_{2}$ . (B) Similarity-matching model. The perceptual branch prompts an LLM with mixture context, embeds generated descriptions, and forms a target perceptual RSM. The molecular branch maps mixture composition to a model RSM; the dashed module is a learned molecule embedding. (C) Held-out RSA correlation, $\rho\left(S^{\mathrm{comp}}_{\mathrm{test}},S^{\mathrm{perc}}_{\mathrm{test}}\right)$ , for three types of kernels. The colors represent the source of perceptual targets. Shuffled values are calculated from randomly permuting the perceptual data before training and then calculating the held out RSA correlations. In the case of diagonal and low-rank kernels, this means the kernels were retrained with shuffled labels. (D) Validation-selected dimension $k$ for the low-rank kernel fits. $\dagger\!$ One fit with $k=1000$ was excluded. Bars show means across runs and splits and error bars indicate 95 $\%$ CI.

Figure 3: Learned molecular geometry and alignment with external molecular features. (A) Example learned molecule–molecule cosine similarity matrix, using a subset of the most frequent training-set molecules ordered by hierarchical clustering (B) Alignment to GS-LF data across different prompts. In calculating the alignment scores, perceptual RSM for the GS-LF data are derived either from word embedding averaging or from binary overlap of the available descriptors between two molecules. Gray points denote alignments after permuting the correspondence between molecules in the learned and GS–LF representations. A separate control that shuffled perceptual labels before model training produced a similarly near-zero alignment. (C) Alignment to GS-LF after filtering molecules based on their geometric stability across fits. The threshold on stability is varied leading to a lower number of included molecules from right to left. (D) Alignment to GS-LF after projection of RSMs to restricted subspaces. Moving from right to left progressively restricts the representations to better-conditioned directions. In (C) and (D), shading denotes crossed-bootstrap $95\%$ CI for the observed curves and $95\%$ permutation-null intervals around the dashed shuffled curves. In all the panels, we included molecules with prevalence of $\geq 10$ across mixtures, and a consensus RSM was created from all the fits before alignment.

Figure 4: Sensory organization and molecular composition alignment of language-derived percepts (A) Similarity among perceptual representations derived from different sensory prompts and mixture-name embeddings, shown for the three embedding methods. Each heatmap entry is a dimension-matched, reliability-adjusted Pearson correlation between the corresponding mixture RSMs. (B) Alignment to GS–LF vs. alignment to molecular composition across sensory and semantic targets. (C) Alignment to GS–LF vs. the effective dimensionality of perceptual targets, quantified using the participation ratio. Dimension matching, reliability adjustment, and participation-ratio calculations are described in Methods IV.6 . (D,E) Alignment to GS–LF after molecular filtering and subspace restriction. These panels are similar to Fig. 3 (C,D), but are plotted for all sensory and semantic targets (word embedding, 10 adjectives).

Figure 5: Predicting mixture perception from external molecular features (A) Zero-shot alignment between mixtures created from features from GS–LF binary, OpenPOM, MolFormer, or Mordred 2D, and perceptual geometries. Mixtures created from all features demonstrate higher than chance alignment to perceptual geometry, but their alignment is significantly lower than the one learned from binary molecular compositions (dashed line). (B) Ridge-adapted alignment using external features only. Gray bars show molecule-row-shuffled controls. Error bars and shaded bands indicate $95\%$ crossed-bootstrap confidence intervals; dashed lines show learned $E$ in A and ridge-adapted binary identity in B. (C) Held-out mixture-perception RSA for identity-only, external-only, identity-plus-external, and identity-plus-shuffled-external models. (D) The corresponding change in RSA relative to the identity-only model. Shading indicates crossed-bootstrap 95% confidence intervals across five runs and five data splits. All data in this figure are based on the Llama 10-adjective target representation.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Condition	Prompt / Output
Fixed 10-adjective smell	System: You are a connoisseur and a sommelier, an expert in describing smells and flavors in exquisite detail. User: Use 10 adjectives to describe the smell of pistachio nut (pistacia vera) (roasted). Return exactly 10 adjectives as a single comma-separated line. Do not use numbering, bullets, line breaks, explanations, or any text other than the adjectives. Output: Rich, buttery, nutty, earthy, sweetly aromatic, toasted, slightly smoky, floral, herbal, complex
Fixed 10-adjective taste	System: You are a chef, gastronome, and tasting expert, skilled at describing tastes and flavors in exquisite detail. User: Use 10 adjectives to describe the taste of pistachio nut (pistacia vera) (roasted). Return exactly 10 adjectives as a single comma-separated line. Do not use numbering, bullets, line breaks, explanations, or any text other than the adjectives. Output: Nutty, rich, buttery, slightly sweet, earthy, herbaceous, slightly bitter, crunchy, toasty, nutty
Fixed 10-adjective visual	System: You are an expert observer. Provide detailed visual descriptions. User: Use 10 adjectives to describe what the appearance of pistachio nut (pistacia vera) (roasted) looks like. Return exactly 10 adjectives as a single comma-separated line. Do not use numbering, bullets, line breaks, explanations, or any text other than the adjectives. Output: Light, brown, irregular, smooth, rounded, greenish-yellow, mottled, speckled, warty, tan
Fixed 10-adjective texture	System: You are a materials expert and haptics specialist, skilled at describing textures and tactile sensations in exquisite detail. User: Use 10 adjectives to describe the texture or feel of pistachio nut (pistacia vera) (roasted). Return exactly 10 adjectives as a single comma-separated line. Do not use numbering, bullets, line breaks, explanations, or any text other than the adjectives. Output: crunchy, nutty, brittle, rough, grainy, slightly-sweet, earthy, buttery, smooth, aromatic
Fixed 10-adjective color	System: You are a color expert and visual designer, skilled at describing colors, hues, tones, and palettes in exquisite detail. User: Use 10 adjectives to describe the typical color or color palette of pistachio nut (pistacia vera) (roasted). Return exactly 10 adjectives as a single comma-separated line. Do not use numbering, bullets, line breaks, explanations, or any text other than the adjectives. Output: muted, beige, pale, golden, creamy, soft-focus greenish-beige, muted green, olive, khaki, warm
Free-form smell	System: You are a connoisseur and a sommelier, an expert in describing smells and flavors in exquisite detail. User: Describe the smell of pistachio nut (pistacia vera) (roasted) in natural language, using no more than 80 words. Mention the most salient odor qualities, but do not force the answer into a list. Finish the response with a complete sentence and terminal punctuation. Output: Roasted pistachio nut exudes a rich, velvety fragrance with a deep, earthy undertone, reminiscent of damp soil and moss. The aroma is complemented by a subtle green note, like the freshness of springtime foliage, and a nutty, toasted quality that hints at the nut’s roasted depth. Overall, the scent is complex, alluring, and utterly inviting, beckoning one to savor the delight of freshly cracked pistachios.

Appendix

Table 1: Example prompts and model outputs for different conditions

Molecule	SMILES	Geometry reliability
(E)-3-hexen-1-ol	CCC=CCCO	0.6525
methyl 2-methylpropanoate (=methyl isobutyrate)	COC(=O)C(C)C	0.6448
(Z)-3-hexenal	CCC=CCC=O	0.6400
trimethylamine	CN(C)C	0.6398
(E)-2-hexen-1-ol	CCCC=CCO	0.6385
cis-linalool oxide (5)	C=CC1(C)CCC(C(C)(C)O)O1	0.6304

Appendix

Table 2: Top 20 molecules ranked by empirical geometry-profile reliability for smell word-embedding 10 adjectives.

Figure S1: VCF data statistics. (A) Distribution of the number of components per mixtures. (B) Distribution of molecular prevalence across mixtures. Molecular prevalence is the number of mixtures a molecule appears in. (C) Singular-value spectrum of the column-centered binary mixture-by-molecule composition matrix $X$ .

Figure S2: Across-splits mixture coverage and similarity. (A)-(B) Distribution of nearest neighbor similarity of train-to-train and test-to-train, as measured by Jaccard (A) and Cosine similarity (B). (C) Histogram of mixture molecular coverage between test and train. For each mixture, molecular coverage $c_{m}$ was calculated as $c_{m}=\sum_{p}x_{mp}\,\mathbf{1}\{p\in\mathcal{P}_{\mathrm{train}}\}\,/\,\sum_{p}x_{mp}.$ This measures to what extent molecules of a given mixture appeared in the training. In these plots, train/validation/test were split randomly following 60/20/20 percentiles.

Figure S3: Most frequent mixture descriptors across sensory prompts. For each sensory target (10 requested adjectives), descriptor counts were pooled across five independent generation runs and divided by five.

Figure S4: Held-out RSA correlation for different numbers of prompted adjectives. Curves show means over multiple split and run seeds; shaded area shows 95% CIs.

Figure S5: Alignment learned molecular kernels to external molecular feature spaces. Bars show Pearson correlations; error bars indicate $95\%$ crossed-bootstrap CIs, and gray markers show shuffled controls. Analyses include molecules with prevalence $\geq 10$ .

Figure S6: Held-out composition-alignment across LLaMA layers. For each given layer, the perceptual data was derived from hidden state of the network at that layer and then aligned to the molecular composition using the low-rank kernels. Held-out Pearson RSA correlations are shown for smell (left) and visual (right) perceptual targets. Curves compare representations obtained by averaging across generated-token embeddings (solid lines) or using the final generated-token embedding (dashed lines), for 10-adjective (dark blue) and free-form (light blue) responses. The green dashed line indicates the mean performance of the 10-adjective word-embedding reference; shading denotes its 95% crossed-bootstrap confidence interval across five generation runs and five mixture-split seeds.

Figure S7: Additional results on sensory organization and molecular composition alignment of language-derived percepts (A) Effective dimensionality of each target representation. We quantified effective dimensionality using the participation ratio, $D_{\mathrm{PR}}=(\sum_{i}\lambda_{i})^{2}/\sum_{i}\lambda_{i}^{2}$ , where $\lambda_{i}$ are the eigenvalues of the centered covariance matrix. Dashed line indicates the dimensionality of the binary mixture-composition matrix. (B) Composition alignment to each sensory target using low-rank kernels. (C) Alignment of learned molecular geometry to GS–LF word-embedding geometry. Bars show means and error bars indicate 95% CIs.

Figure S8: Gaussian random molecular features and mixture-perception alignment Held-out RSA correlation as a function of random-feature dimension $k$ , before and after ridge adaptation. Shading denotes $\pm 1$ SEM across five runs; dashed lines show the identity-kernel and trained low-rank- $E$ baselines.

Figure S9: Prompt stability and response confidence for mixture and single-molecule prompts. (A) Geometric stability across prompt-generation runs, quantified as the mean Pearson correlation between representational similarity matrices constructed from word-embedding representations or Llama activations. Each colored point represents one of $B=10$ repetitions using 1,000 pair-valid items; gray points show the corresponding label-permutation controls. (B) Item-wise stability, quantified as the mean same-item cosine similarity across valid generation-run pairs for word-embedding and Llama representations. (C) Model-reported response confidence, averaged across the two available confidence runs. After generating the adjectives, the model received a second prompt: Based on your knowledge, rate your confidence that your preceding adjectives accurately describe [the target]. Use an integer from 0 to 100. Evaluate perceptual accuracy, not formatting or writing quality. Return only the integer. Geometric and item-wise stability measures were calculated over 5 independent prompting runs. Filled boxes span the interquartile range ( $Q_{1}$ – $Q_{3}$ ), black lines indicate medians, open circles indicate means, and whiskers span the 10th–90th percentiles. Colors distinguish Mixture (smell prompt), Molecule (chemical prompt), and Molecule (smell prompt).

Molecular Representation Learning

Molecular representation learning encodes chemical structures and context for prediction, generation, and discovery across chemistry and biology.

See all 205 papers

Mar 8, 2026

Benchmark for Assessing Olfactory Perception of Large Language Models

Here we introduce the Olfactory Perception (OP) benchmark, designed to assess the capability of large language models (LLMs) to reason about smell. The benchmark contains 1,010 questions across eight task categories spanning odor classification, odor primary descriptor identification, intensity and pleasantness judgments, multi-descriptor prediction, mixture similarity, olfactory receptor activation, and smell identification from real-world odor sources. Each question is presented in two prompt formats, compound names and isomeric SMILES, to evaluate the effect of molecular representations. Evaluating 21 model configurations across major model families, we find that compound-name prompts consistently outperform isomeric SMILES, with gains ranging from +2.4 to +18.9 percentage points (mean approx +7 points), suggesting current LLMs access olfactory knowledge primarily through lexical associations rather than structural molecular reasoning. The best-performing model reaches 64.4% overall accuracy, which highlights both emerging capabilities and substantial remaining gaps in olfactory reasoning. We further evaluate a subset of the OP across 21 languages and find that aggregating predictions across languages improves olfactory prediction, with AUROC = 0.86 for the best performing language ensemble model. LLMs should be able to handle olfactory and not just visual or aural information.

Eftychia Makri, Nikolaos Nakis, Laura Sisson +4

Department of Electrical Engineering, Yale University, New Haven, CT, USA · Human Nature Lab, Yale University, New Haven, CT, USA · Patina, New York, NY, USA +1

LLM Evaluation

Ask PDF

Aug 10, 2026

Decoding Mixture Perception through Computational Modeling of Component Interactions

Olfaction played an indispensable role throughout human evolution and civilization. Even in the contemporary era of advanced technology, olfaction remains a critical channel for person to conduct danger discrimination, emotional experience, and memory formation. However, most substances in nature exist as multi-molecule mixtures. The complexity of mixture compositions, as well as concentration dependent saturation effects and receptor specific activation thresholds, pose substantial challenges in identifying olfactory characteristics. In this study, we proposed a novel bio inspired deep learning framework for accurate odor perception recognition of mixtures. We robustly constructed neural response curves for molecule-receptor interactions, and developed a fusion strategy that integrates attention-weighted multi-receptor curves with concentration-dependent multi-molecule curves, replicating the competitive activation and synergistic integration of mixture components. Furthermore, by comparing the consistency of response curve patterns, the model can transfer knowledge from the semantically rich space of molecular associations to guide recognition of mixture perception characteristics. Therefore, we established a complete computational pathway from chemical blending, neural encoding, to perceptual formation. Finally, we conducted comprehensive evaluation, and results demonstrated exceptional superiority, achieving an accuracy of 92.2%. Consequently, our work provides a generalizable solution to the long standing mixture perception challenge. More importantly, it can be integrated into embodied cognitive systems to enhance the agents perceptual and interactive capabilities in complex scenarios.

Fei Wang, Xiaoya Xie, Junfei Liu +6

College of Biomedical Engineering & Instrument Science, Zhejiang University, Hangzhou, 310012, China · College of Automation Engineering, Nanjing University of Aeronautics and Astronautics, Nanjing 210016, China

Neural Encoding

Ask PDF

Apr 2, 2026

AromaGen: Interactive Generation of Rich Olfactory Experiences with Multimodal Language Models

Smell's connections with food, memory, and social experience have motivated researchers to bring olfaction into interactive systems. Multimodal AI opens new possibilities for generating smells from natural language, yet it remains unclear whether pretrained models encode sufficient olfactory knowledge to translate language into perceptible compositions. We present AromaGen, an AI-powered 12-channel wearable olfactory system that maps free-form natural-language descriptions to 12 base odorants selected to cover a semantically derived olfactory space, while supporting iterative refinement through natural-language feedback. In a between-subjects study (N=60) across 50 real-world benchmark smells, participants distinguished AromaGen-generated target smells above chance in a three-alternative forced-choice task, with retrieval-augmented AI composition performing comparably to human-expert composition and zero-shot AI composition. Natural-language feedback significantly improved the perceived similarity of both AI- and human-expert compositions. Our findings demonstrate the feasibility of using pretrained multimodal AI, with human aroma composition data, to generate perceptible olfactory experiences from natural language.

Awu Chen, Silang Wang, Yunge Wen +6

MIT Media Lab, Cambridge, MA, USA · University of Washington, Seattle, WA, USA · Harvard Graduate School of Design, Cambridge, MA, USA +3

Multimodal Generation Multimodal Large Language Models

Ask PDF

Abstract

Molecular Representation Learning

Explore similar work

Benchmark for Assessing Olfactory Perception of Large Language Models

Decoding Mixture Perception through Computational Modeling of Component Interactions

AromaGen: Interactive Generation of Rich Olfactory Experiences with Multimodal Language Models