NOSE: Neural Olfactory-Semantic Embedding with Tri-Modal Orthogonal Contrastive Learning

Yanyi Su; Hongshuai Wang; Zhifeng Gao; Jun Cheng

cs.CLApr 12, 2026

NOSE: Neural Olfactory-Semantic Embedding with Tri-Modal Orthogonal Contrastive Learning

Authors: Yanyi Su, Hongshuai Wang, Zhifeng Gao, Jun Cheng

Organizations: State Key Laboratory of Physical Chemistry of Solid Surface, College of Chemistry and Chemical Engineering, Xiamen University, Xiamen, China · DP Technology · Laboratory of AI for Electrochemistry (AI4EC), Tan Kah Kee Innovation Laboratory (IKKEM), Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China

Abstract

Olfaction lies at the intersection of chemical structure, neural encoding, and linguistic perception, yet existing representation methods fail to fully capture this pathway. Current approaches typically model only isolated segments of the olfactory pathway, overlooking the complete chain from molecule to receptors to linguistic descriptions. Such fragmentation yields learned embeddings that lack both biological grounding and semantic interpretability. We propose NOSE (Neural Olfactory-Semantic Embedding), a representation learning framework that aligns three modalities along the olfactory pathway: molecular structure, receptor sequence, and natural language description. Rather than simply fusing these signals, we decouple their contributions via orthogonal constraints, preserving the unique encoded information of each modality. To address the sparsity of olfactory language, we introduce a weak positive sample strategy to calibrate semantic similarity, preventing erroneous repulsion of similar odors in the feature space. Extensive experiments demonstrate that NOSE achieves state-of-the-art (SOTA) performance and excellent zero-shot generalization, confirming the strong alignment between its representation space and human olfactory intuition.Code and data are available at https://github.com/Xianyusyy/NOSE

Figures & tables

Modality	Input Example
Molecule	CC1CCCCCCCCCCCCC(=O)C1 (Muscone)
Receptor	MRENNQSSTLEFILLGVTG... (OR5A2, 312 amino acids)
Description	“musk; powdery; sweet; floral”

Table 1: Representative input examples for the three pre-training modalities. Molecule and receptor inputs correspond to Muscone and its cognate receptor OR5A2, respectively.

Figure 1: Subscripts $r$ and $d$ denote receptor and description modalities, respectively. (Left) Multimodal Orthogonal Pre-training: Molecular representations $z_{mol}$ are extracted by a frozen Uni-Mol encoder, receptor embeddings $z_{rec}$ are obtained from ESM-2 with a trainable projection layer, and odor semantic descriptions $z_{desc}$ are extracted by LoRA-finetuned Qwen3 Embedding after LLM-based weak positive augmentation. The molecular embedding is decomposed via dual adapters into a receptor-aligned component ( $a_{r}$ ) and a description-aligned component ( $a_{d}$ ), which are then orthogonalized through Gram-Schmidt to yield $z_{r}$ and $z_{d}$ . Training objectives include: receptor-molecule InfoNCE loss, description-molecule soft-weighted InfoNCE loss (with positive/weak-positive/negative sample weights of 1.0/0.5/0.0), intra-modal InfoNCE loss, and orthogonality constraints among $z_{mol}$ , $a_{r}$ , and $a_{d}$ . (Right) Downstream Inference: Inference requires only the molecular encoder and adapters, with the final representation $Z=w_{1}\cdot z_{mol}+w_{2}\cdot a_{r}+w_{3}\cdot a_{d}$ , supporting both pure substance and mixture perception tasks.

Category	Task	Data Format	Source (Size)
Pretraining Corpora	Contrastive	SMILES–Receptor Sequence pairs	Lalis et al. , etc. (3,877)
	Contrastive	SMILES–Odor Description pairs	Arctander , etc. (88,512)
	Contrastive	SMILES–Odor Description pairs	LLM-augmented * (2,567,558)
Basic Perception	Threshold	SMILES $\to$ $[-2.78,6.11]$	Abraham et al. (268)
	Pleasantness	SMILES $\to$ $[0,100]$	Keller and Vosshall (474)
	Pleasantness	SMILES $\to$ $[-1,1]$	Sagar et al. (160)

Table 2: Overview of pretraining corpora and downstream evaluation tasks. Descriptor Classification is a multi-label classification task, while Descriptor Strength involves multi-dimensional regression. ∗ We leverage an LLM to scale up the pretraining corpora by orders of magnitude.

	Thresholds	Pleasantness		Intensity
Dataset	Abraham	Keller	Sagar	Keller	Sagar	Ravia
GIN	0.72(0.02)	0.30(0.04)	0.08(0.06)	0.06(0.14)	-0.06(0.27)	0.19(0.11)
GCN	0.22(0.09)	0.43(0.01)	0.12(0.27)	0.11(0.05)	0.30(0.04)	0.19(0.01)
Morgan	0.60(0.18)	0.51(0.03)	0.21(0.06)	0.05(0.02)	0.23(0.10)	0.10(0.19)
AttentiveFP	0.59(0.35)	0.52(0.05)	0.23(0.05)	0.33(0.01)	0.23(0.05)	0.47(0.00)
Uni-Mol	0.78(0.05)	0.68(0.02)	0.14(0.34)	0.27(0.03)	0.37(0.05)	0.31(0.02)

Table 3: Performance comparison on basic perceptual attribute prediction tasks (Pearson

\uparrow

	Descriptor Classification			Descriptor Strength
Dataset	GS-LF			Keller		Sagar
Metric	AUC $\uparrow$	AUPRC $\uparrow$	MCC $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$
GIN	0.856(0.001)	0.270(0.008)	0.064(0.007)	6.528(0.139)	0.128(0.007)	0.358(0.005)	-0.034(0.016)
GCN	0.858(0.001)	0.279(0.001)	0.084(0.003)	6.247(0.031)	0.171(0.006)	0.407(0.098)	0.108(0.085)
Morgan	0.850(0.002)	0.320(0.002)	0.222(0.023)	6.483(0.015)	0.244(0.023)	0.395(0.018)	0.008(0.076)
POM	0.868(0.002)	0.336(0.005)	0.233(0.023)	6.304(0.258)	0.314(0.039)	0.405(0.113)	0.065(0.062)

Table 4: Performance comparison on semantic description prediction tasks.

	Mixture Pleasantness				Mixture Intensity
Metric	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GCN	-14.30(9.52)	3.69(1.72)	0.33(0.42)	17.48(10.88)	-0.46(0.50)	0.52(0.09)	0.49(0.07)	0.40(0.14)
GIN	-0.10(0.12)	0.88(0.06)	0.47(0.03)	1.25(0.14)	-2.82(0.30)	0.86(0.02)	0.01(0.14)	1.06(0.08)
Morgan	0.50(0.19)	0.60(0.11)	0.81(0.01)	0.57(0.21)	0.14(0.31)	0.40(0.08)	0.58(0.06)	0.24(0.08)
AttentiveFP	0.49(0.07)	0.62(0.05)	0.74(0.02)	0.58(0.08)	0.26(0.06)	0.36(0.01)	0.60(0.03)	0.20(0.02)
POM	0.56(0.03)	0.56(0.00)	0.77(0.02)	0.51(0.03)	0.35(0.06)	0.36(0.02)	0.60(0.03)	0.18(0.02)

Table 5: Performance comparison on mixture perception prediction tasks.

Settings	Variant	Avg Rank
Multi- modality	Molecule only	3.3226
	Receptor only	2.8065
	Description only	2.7742
	Tri-modal (ours)	1.0968
Orthogonal Module	Hard + Soft $\lambda$ =0.5	5.1613
	Hard + Soft $\lambda$ =0.1	5.0645

Table 6: Ablation study results.

Task	N	Hits@1	Hits@5	Hits@10	Hits@20	Hits@50
Odor Retrieval (Zero-shot)	27	14.8%	33.3%	51.9%	66.7%	85.2%
Odor Retrieval (Strict Zero-shot)	15	6.7%	6.7%	13.3%	33.3%	53.3%
Receptor Retrieval (Activated)	13	0%	61.5%	69.2%	84.6%	84.6%

Table 7: Zero-shot cross-modal retrieval performance (SMILES → descriptors/receptors). N : number of test samples. Hits@k : percentage of ground-truth targets ranked within top k. Zero-shot : pairs unseen during training, though molecules may appear in other pairs. Strict Zero-shot : molecules entirely absent from training. For Activated, higher Hits@k is better (target receptor should rank high). For Inactivated, lower Hits@k is better (non-binding receptor should rank low).

	Qwen3 Embedding
	Original	LoRA	LoRA+Head
MRR $\uparrow$	0.0102	0.1857	0.2072
Hits@1 (%) $\uparrow$	0.0	6.5	6.5
Hits@5 (%) $\uparrow$	0.0	25.8	32.3
Hits@10 (%) $\uparrow$	3.2	45.2	54.8
Hits@20 (%) $\uparrow$	3.2	71.0	90.3

Table 8: Compositional retrieval performance on 31 manually designed arithmetic queries (e.g., Lemon

-

Sour

\rightarrow

[citrus, orange, …]). Original : Qwen3 Embedding without fine-tuning; LoRA : with LoRA fine-tuning; LoRA+Head : LoRA combined with contrastive learning projection head. MRR: Mean Reciprocal Rank (

1/\text{rank}

Appendix figures & tables39 assets

Supplementary material from the paper’s appendix.

Appendix

Hyperparameter	Search Space
Batch size	[4, 8, 16, 32]
	([32, 64, 128, 256] * )
Learning rate	[5e-4, 3e-4, 1e-4, 7e-5, 5e-5, 5e-6]
Patience	[5, 10, 15, 20, 25]
Early stop metric	[R 2 , MAE, Pearson, MSE]
	([AUC, AUPRC, MCC] * )

Appendix

Table 9: Hyperparameter search space for downstream tasks. * For classification tasks.

Task	Train	Val	Test	Batch Size	Learning Rate	Patience	Early Stop Metric
Thresholds	160	54	54	8	5e-4	20	Pearson
Pleasantness (Keller)	379	47	48	4	1e-4	15	MSE
Pleasantness (Sagar)	128	16	16	4	1e-4	25	Pearson
Intensity (Keller)	379	47	48	16	5e-5	25	MSE
Intensity (Sagar)	128	16	16	16	5e-5	20	MSE
Intensity (Ravia)	198	25	25	8	5e-6	15	R 2

Appendix

Table 10: Dataset splits and selected hyperparameters for NOSE on each downstream task.

	Rec $\rightarrow$ Desc			Desc $\rightarrow$ Rec
Metric	Rand.	Direct	Bridge	Rand.	Direct	Bridge
MRR	0.006	0.058	0.056	0.010	0.110	0.199
Hits@1(%)	0.1	0.5	2.4	0.2	4.0	6.7
Hits@5(%)	0.5	5.2	4.4	0.8	15.7	33.2
Hits@10(%)	0.9	16.3	8.7	1.6	30.0	44.4
Hits@20(%)	1.8	42.9	19.4	3.1	43.7	52.9

Appendix

Table 11: Cross-modal retrieval via bridge molecules. “Rand.” denotes the random baseline. “Direct” queries one modality against the other without molecular mediation. “Bridge” retrieves via the molecular hub (query

\rightarrow

molecule

\rightarrow

target).

$k$	Uni-Mol	NOSE	Improvement
1	0.775	0.843	+8.9%
5	0.718	0.809	+12.6%
10	0.692	0.794	+14.7%
20	0.666	0.782	+17.4%
50	0.632	0.765	+20.9%
100	0.607	0.748	+23.3%

Appendix

Table 12: Neighborhood consistency (Precision@

k

). Higher values indicate that nearby molecules in the embedding space share odor descriptors.

	Uni-Mol		NOSE
Cosine Range	Jaccard	#pairs	Jaccard	#pairs
$[-0.2,\,-0.1)$	–	0	0.023	144,579
$[-0.1,\,0.0)$	–	0	0.029	405,096
$[0.0,\,0.1)$	–	0	0.039	558,295
$[0.1,\,0.2)$	–	0	0.055	448,443
$[0.2,\,0.3)$	–	0	0.075	244,121

Appendix

Table 13: Embedding distance vs. perceptual similarity. For each cosine-similarity bin we report the mean Jaccard similarity of shared odor descriptors and the number of molecule pairs. “–” indicates no pairs fall in the bin.

Task	B (NOSE)	C (Dim-Split)	$\Delta$
Threshold	0.652 $\pm$ 0.072	0.425 $\pm$ 0.206	+0.227
Keller Regression	0.075 $\pm$ 0.040	$-$ 0.037 $\pm$ 0.066	+0.112
Intensity (Keller)	0.156 $\pm$ 0.020	$-$ 0.048 $\pm$ 0.108	+0.204
Pleasantness (Keller)	0.488 $\pm$ 0.074	0.404 $\pm$ 0.134	+0.084
Sagar Regression	$-$ 0.305 $\pm$ 0.106	$-$ 0.347 $\pm$ 0.087	+0.042
Intensity (Sagar)	0.120 $\pm$ 0.109	$-$ 0.100 $\pm$ 0.132	+0.220

Appendix

Table 14: Per-task comparison of NOSE (Config B, orthogonal injection + gated fusion) vs. dimension splitting (Config C, concatenation without orthogonality).

\Delta

= B

-

C. Primary metric is

R^{2}

except GS-LF (AUC).

Task	Gate Gain (B $-$ A)	Concat Gain (D $-$ C)
Threshold	+0.146	+0.163
Keller Regression	+0.074	$-$ 0.043
Intensity (Keller)	+0.106	+0.113
Pleasantness (Keller)	+0.145	+0.004
Sagar Regression	+0.024	$-$ 0.054
Intensity (Sagar)	+0.258	$-$ 0.046

Appendix

Table 15: Per-task effect of adding orthogonal constraints, stratified by fusion strategy. Gate Gain = Config B

-

Config A; Concat Gain = Config D

-

Config C.

Task	No Orth. std	NOSE std	Change
Threshold	0.254	0.072	$\downarrow$ 71.7%
Pleasantness (Sagar)	0.078	0.020	$\downarrow$ 74.4%
Intensity (Keller)	0.077	0.010	$\downarrow$ 87.0%
Intensity (Sagar)	0.253	0.109	$\downarrow$ 56.9%
Intensity (Ravia)	0.107	0.025	$\downarrow$ 76.6%
GS-LF	0.025	0.010	$\downarrow$ 60.0%

Appendix

Table 16: Cross-seed variance (std of primary metric over 3 seeds). Lower is better. “Change” shows the relative reduction from No Orth. to NOSE.

Task	$w_{\text{mol}}$	$w_{\text{rec}}$	$w_{\text{desc}}$	max std
Keller Strength	0.900	0.061	0.039	0.008
Abraham Threshold	0.553	0.221	0.226	0.015
Sagar Strength	0.390	0.310	0.300	0.012
Mixture Intensity	0.420	0.350	0.230	0.011
GS-LF Classification	0.310	0.280	0.410	0.014

Appendix

Table 17: Fusion weights (mean over 5 seeds) for representative tasks. “max std” is the largest standard deviation among the three weights.

Fraction	$w_{\text{mol}}$	$w_{\text{rec}}$	$w_{\text{desc}}$
Keller Strength (max dev. 0.003)
100%	0.900	0.061	0.039
50%	0.898	0.063	0.039
25%	0.897	0.064	0.039
Sagar Strength (max dev. 0.021)
100%	0.390	0.310	0.300

Appendix

Table 18: Fusion weight stability across data fractions. “max dev.” denotes the maximum absolute deviation from the 100% setting among all three weights.

Setting	Silhouette	D-B	C-H
Qwen3 Emb. (original)	$-0.003$	9.011	2.72
LoRA+Head (w/ weak pos.)	0.102	2.504	28.12
LoRA+Head (w/o weak pos.)	0.022	5.889	5.28

Appendix

Table 19: Clustering quality of descriptor embeddings across three olfactory-semantic groups (274 descriptors). D-B = Davies-Bouldin, C-H = Calinski-Harabasz.

Figure 2: Vector Space Visualization. For a given SMILES input, NOSE generates vectors in three orthogonal spaces: molecular representation, receptor representation, and description representation. Each column corresponds to the same representation space. Each row is color-coded according to different attributes. The first row is colored by molecular scaffolds: Decalin, Cyclododecane, and Thiophene. The second row is colored by receptor activation: A0A024RCH9, Q8NH74, and A0A126GWS0. The third row is colored by odor descriptor: alliaceous and minty.

Fruity (111 terms)	Green / Herbal (80 terms)	Gourmand / Sweet (83 terms)
apple, apple cooked apple, apple dried apple, apple green apple, apple peel, apple skin, apricot, banana, banana peel, banana ripe banana, banana unripe banana, berry, berry ripe berry, blackberry, blueberry, cherry, cherry maraschino cherry, concord, concord grape, cranberry, currant, currant black currant, date, dried fruit, dry fruit, durain, durian, elderberry, fig, fruit, fruit dried fruit, fruit overripe fruit, fruit ripe fruit, fruit skin, fruit tropical fruit, fruity, gooseberry, grape, grape skin, green peach, green pear, guava, jackfruit, jam, jammy, juicy, juicy fruit, kiwi, loganberry, lychee, mango, maraschino, melon, melon rind, melon unripe melon, non-citrus fruity, overripe, overripe fruit, papaya, passion, passion fruit, peach, pear, pear skin, pineapple, plum, plum skin, pomegranate, prune, pulpy, pulpy fruit, quince, raisin, raspberry, red berry, rhubarb, ripe, ripe fruit, starfruit, strawberry, tropical, tropical-fruit, tutti frutti, tutty-fruity, unripe, unripe banana, unripe fruit, watermelon, watermelon rind, bergamot, citral, citralva, citric, citronella, citronellal, citronellol, citrus, citrus peel, citrus rind, grapefruit, grapefruit peel, grapfruit, hesperidic, lemon, lemon peel, lemongrass, lime, limonene, mandarin, orange, orange bitter orange, orange peel, orange rind, tangerine, zesty	angelica, armoise, artemisia, basil, bay, bayleaf, buchu, celery, chamomile, chervil, cilantro, clary, clary sage, coriander, cress, cut grass, davana, dill, eucalyptol, eucalyptus, fennel, fern, foliage, fresh cut grass, galbanum, grassy, grassy (fresh, sweet), grassy (green, sharp), green, green leaf, hay, hay new mown hay, herb, herba-, herbaceous, herbal, leaf, leafy, lettuce, lovage, marjoram, menthol, mentholic, mint, minty, minty tea, mown, new mown hay, origanum, parsley, peppermint, petitgrain, plant, plants, rosemary, rue, sage, sage clary sage, sassafras, sassafrass, spearmint, spinach, stalk, stem, tagette, tarragon, tea, tea black tea, tea green tea, tea rose, thyme, tomato leaf, vegetation, verbena, watercress, weed, weedy, wintergreen, wormwood	acetoin, almond, baked, bakery, biscuit, bonbon, bread, bread baked, bread crust, bread rye bread, bready, brown sugar, bubble gum, bubblegum, burnt sugar, butterscotch, cakes, candy, caramel, caramelic, caramellic, cereal, chocolate, chocolate dark chocolate, coco, cocoa, coconut, coffee, coffee roasted coffee, cognac, cookie, cookies, cotton candy, coumarin, coumarinic, custard, dark chocolate, food, food like, gourmand, graham cracker, grain, grain toasted grain, grains, honey, honeydew, iactonic, lactone, lactonic, malt, malty, maple, marshmallow, marzipan, molasses, popcorn, powdery, praline, preserves, rum, rummy, rye, rye bready, sugar, sugar brown sugar, sugar burnt sugar, sweet, sweet (medicinal), syrup, toasted, toasted grain, toasty, toffee, tonka, vanilla, vanillin, whiskey, whisky, yeast, yeasty

Appendix

Table 20: Selected Olfactory Term Categories.

Figure 3: (a) Qwen3 Embedding without LoRA fine-tuning lack olfactory semantics. (b) After contrastive learning, Qwen3 Embedding acquire olfactory semantics.

		Qwen3 Embedding
Query	Expected Answer	Original	LoRA	LoRA+Head
mushroom + wet	foul	64	9	9
	fungal	66	11	11
	earthy	685	22	16
	bark	122	143	39
meat + smoke	bacon	9	1	1

Appendix

Table 21: Compositional retrieval results. Numbers indicate the rank of expected odor descriptors in the predicted list (lower is better).

Name	CAS	Description (PubChem)	Retrieved Descriptors (Rank, Top%)
Zero-shot
Bromobenzene	108-86-1	AROMATIC ODOR; Pungent odor	penetrating (14/1086, 1.3%) irritating (17/1086, 1.6%) pungent (111/1086, 10.2%) aromatic (258/1086, 23.8%)
Iodoform	75-47-8	Characteristic, disagreeable odor; pungent, disagreeable odor	irritating (11/1086, 1.0%) penetrating (13/1086, 1.2%) repulsive (46/1086, 4.2%) pungent (117/1086, 10.8%) disagreeable (163/1086, 15.0%)
Cyclohexylamine	108-91-8	Strong, fishy, amine odor	amine (1/1086, 0.1%) ammoniacal (2/1086, 0.2%) ammonia (3/1086, 0.3%) dead animal (6/1086, 0.6%) rotten fish (11/1086, 1.0%) fishy (21/1086, 1.9%)
2,4-Dinitrotoluene	121-14-2	Slight odor	odorless (1/1086, 0.1%) slight (2/1086, 0.2%) neutral (3/1086, 0.3%) weak (5/1086, 0.5%)
D-Glucose	-	Odorless	odorless (1/1086, 0.1%) neutral (5/1086, 0.5%) slight (7/1086, 0.6%) weak (9/1086, 0.8%)

Appendix

Table 22: Detailed zero-shot SMILES-to-odor retrieval results. Rank indicates the position among 1,086 candidate descriptors (lower is better). Results in the top 5% are shown in bold .

Name	CAS	Gene	Source	(Rank, Top%)
Activated — Macrocyclic Musks (MCM)
Globanone	3100-36-5	OR5A2	Yoshikawa et al. (2022)	(2/636, 0.315%)
Muscenone delta	82356-51-2	OR5A2	Yoshikawa et al. (2022)	(2/636, 0.315%)
Cosmone	259854-70-1	OR5A2	Yoshikawa et al. (2022)	(2/636, 0.315%)
Cyclopentadecanol	4727-17-7	OR5A2	Yoshikawa et al. (2022)	(2/636, 0.315%)
Ambrettolide	7779-50-2	OR5A2	Yoshikawa et al. (2022)	(2/636, 0.315%)

Appendix

Table 23: Detailed zero-shot SMILES-to-receptor retrieval results. Rank% indicates the percentile ranking among 636 candidate receptors. For activated pairs, lower is better (results below 5% in bold ). For inactivated pairs, higher rank% indicates successful rejection.

	Thresholds (Abraham)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GCN	-0.304(0.431)	1.309(0.247)	0.220(0.089)	2.912(0.963)
AttentiveFP	0.102(0.634)	1.073(0.375)	0.591(0.351)	2.006(1.417)
GIN	0.339(0.135)	0.969(0.129)	0.723(0.018)	1.477(0.300)
Morgan	0.360(0.215)	0.924(0.119)	0.596(0.175)	1.431(0.481)
POM	0.565(0.087)	0.731(0.082)	0.788(0.024)	0.972(0.195)

Appendix

Table 24: Basic perceptual attribute prediction: Thresholds.

	Pleasantness (Keller)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GIN	-0.175(0.081)	10.540(0.178)	0.299(0.037)	205.154(14.095)
Uni-Mol	-0.071(0.204)	10.576(1.354)	0.680(0.017)	186.939(35.551)
GCN	0.073(0.025)	8.610(0.128)	0.435(0.007)	161.831(4.433)
Morgan	0.180(0.116)	9.280(0.879)	0.515(0.034)	143.160(20.192)
ChemBERTa	0.120(0.216)	9.194(0.675)	0.649(0.036)	153.589(37.664)

Appendix

Table 25: Basic perceptual attribute prediction: Pleasantness (Keller).

	Pleasantness (Sagar)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GIN	-0.139(0.092)	0.497(0.021)	0.079(0.056)	0.341(0.028)
Morgan	-0.085(0.054)	0.489(0.010)	0.213(0.055)	0.325(0.016)
GCN	-0.053(0.100)	0.465(0.024)	0.116(0.272)	0.316(0.030)
ChemBERTa	-0.028(0.070)	0.475(0.012)	0.152(0.125)	0.308(0.021)
AttentiveFP	-0.008(0.030)	0.465(0.007)	0.233(0.046)	0.302(0.009)

Appendix

Table 26: Basic perceptual attribute prediction: Pleasantness (Sagar).

	Intensity (Keller)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Uni-Mol	-0.470(0.317)	17.189(1.927)	0.270(0.031)	470.285(101.497)
GIN	-0.318(0.165)	16.295(0.820)	0.063(0.141)	421.540(52.643)
GCN	-0.249(0.107)	16.457(0.641)	0.109(0.052)	399.513(34.145)
Morgan	-0.248(0.053)	15.981(0.431)	0.053(0.024)	399.332(16.902)
ChemBERTa	-0.117(0.145)	15.001(1.156)	0.393(0.070)	357.445(46.307)

Appendix

Table 27: Basic perceptual attribute prediction: Intensity (Keller).

	Intensity (Sagar)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GIN	-1.492(1.869)	0.540(0.223)	-0.056(0.266)	0.508(0.381)
Morgan	-0.584(0.191)	0.465(0.026)	0.230(0.102)	0.323(0.039)
ChemBERTa	-0.336(0.136)	0.462(0.022)	0.447(0.082)	0.272(0.028)
AttentiveFP	-0.216(0.201)	0.450(0.048)	0.228(0.055)	0.248(0.041)
Uni-Mol	-0.317(0.102)	0.442(0.022)	0.372(0.051)	0.269(0.021)

Appendix

Table 28: Basic perceptual attribute prediction: Intensity (Sagar).

	Intensity (Ravia)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Morgan	-2.832(4.010)	24.444(17.045)	0.103(0.186)	1062.533(1111.892)
Uni-Mol	-0.407(0.352)	15.176(1.993)	0.307(0.024)	390.301(97.745)
GIN	-0.191(0.172)	14.514(1.228)	0.188(0.114)	330.250(47.774)
GCN	-0.109(0.006)	13.426(0.540)	0.192(0.014)	307.666(1.719)
POM	-0.080(0.057)	12.833(0.297)	0.319(0.055)	299.530(15.798)

Appendix

Table 29: Basic perceptual attribute prediction: Intensity (Ravia).

	GS-LF Multi-label Multi-class Classification
Method	AUC $\uparrow$	AUPRC $\uparrow$	MCC $\uparrow$
GIN	0.856(0.001)	0.270(0.008)	0.064(0.007)
GCN	0.858(0.001)	0.279(0.001)	0.084(0.003)
Morgan	0.850(0.002)	0.320(0.002)	0.222(0.023)
AttentiveFP	0.867(0.004)	0.328(0.012)	0.203(0.026)
POM	0.868(0.002)	0.336(0.005)	0.233(0.023)

Appendix

Table 30: Semantic description prediction on GS-LF dataset.

	Multi-label Regression (Keller)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GIN	-0.229(0.072)	6.528(0.139)	0.128(0.007)	90.903(4.613)
Uni-Mol	-0.178(0.060)	6.741(0.209)	0.330(0.050)	90.373(5.967)
Morgan	-0.087(0.024)	6.483(0.015)	0.244(0.023)	80.531(1.690)
GCN	-0.106(0.011)	6.247(0.031)	0.171(0.006)	80.182(0.243)
POM	-0.125(0.076)	6.304(0.258)	0.314(0.039)	78.649(4.872)

Appendix

Table 31: Semantic description prediction on Keller dataset.

	Multi-label Regression (Sagar)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
POM	-0.835(0.821)	0.405(0.113)	0.065(0.062)	0.281(0.094)
GCN	-0.933(1.067)	0.407(0.098)	0.108(0.085)	0.254(0.076)
GIN	-0.654(0.022)	0.358(0.005)	-0.034(0.016)	0.248(0.006)
Morgan	-0.563(0.273)	0.395(0.018)	0.008(0.076)	0.233(0.019)
Uni-Mol	-0.677(0.218)	0.355(0.009)	0.116(0.042)	0.252(0.002)

Appendix

Table 32: Semantic description prediction on Sagar dataset.

	Mixture Intensity
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GIN	-2.821(0.297)	0.857(0.017)	0.006(0.143)	1.056(0.082)
GCN	-0.455(0.503)	0.516(0.094)	0.491(0.073)	0.402(0.139)
Morgan	0.143(0.307)	0.397(0.081)	0.575(0.063)	0.237(0.085)
AttentiveFP	0.260(0.059)	0.361(0.006)	0.597(0.034)	0.205(0.016)
Uni-Mol	0.319(0.106)	0.355(0.028)	0.637(0.069)	0.188(0.029)

Appendix

Table 33: Mixture prediction tasks: Intensity.

	Mixture Pleasantness
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
GCN	-14.296(9.524)	3.685(1.723)	0.332(0.418)	17.477(10.883)
GIN	-0.096(0.123)	0.879(0.056)	0.473(0.029)	1.252(0.140)
ChemBERTa	0.452(0.074)	0.641(0.060)	0.793(0.009)	0.626(0.084)
AttentiveFP	0.490(0.068)	0.616(0.051)	0.744(0.015)	0.583(0.078)
Morgan	0.498(0.187)	0.599(0.113)	0.814(0.012)	0.574(0.214)

Appendix

Table 34: Mixture prediction tasks: Pleasantness.

	Thresholds (Abraham)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Adapter (Desc: 10.00M, Rec: 10.00M)	0.161(0.616)	1.074(0.421)	0.768(0.049)	1.875(1.375)
Adapter (Desc: 29.17M, Rec: 10.00M)	0.331(0.179)	0.986(0.148)	0.742(0.042)	1.494(0.401)
Inter	0.338(0.446)	0.978(0.374)	0.782(0.063)	1.478(0.995)
Hard + Soft $\lambda$ =1.0	0.440(0.072)	0.884(0.085)	0.749(0.006)	1.252(0.162)
Receptor only	0.444(0.173)	0.909(0.173)	0.791(0.041)	1.242(0.387)

Appendix

Table 35: Ablation study: Basic perceptual attribute prediction: Thresholds.

	Pleasantness (Keller)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
No Orthogonal	0.343(0.078)	7.818(0.487)	0.622(0.046)	114.722(13.697)
Hard + Soft $\lambda$ =1.0	0.354(0.055)	8.079(0.355)	0.623(0.040)	112.703(9.565)
Molecule only	-0.071(0.204)	10.576(1.354)	0.680(0.017)	186.939(35.551)
Description only	0.366(0.110)	7.959(0.877)	0.647(0.063)	110.652(19.235)
Receptor only	0.368(0.104)	8.025(0.690)	0.636(0.102)	110.308(18.106)

Appendix

Table 36: Ablation study: Pleasantness prediction (Keller).

	Pleasantness (Sagar)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Inter + Intra	-0.282(0.336)	0.511(0.047)	0.025(0.245)	0.384(0.101)
Inter	-0.219(0.214)	0.505(0.023)	-0.034(0.285)	0.365(0.064)
Adapter (Desc: 10.00M, Rec: 10.00M)	-0.208(0.152)	0.492(0.036)	-0.127(0.181)	0.362(0.046)
Only Hard	-0.169(0.027)	0.508(0.016)	0.085(0.158)	0.350(0.008)
No Orthogonal	-0.148(0.110)	0.490(0.022)	0.083(0.078)	0.344(0.033)

Appendix

Table 37: Ablation study: Basic perceptual attribute prediction: Pleasantness (Sagar).

	Intensity (Keller)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Molecule only	-0.470(0.317)	17.189(1.927)	0.270(0.031)	470.285(101.497)
Inter	-0.224(0.096)	16.201(0.767)	0.206(0.042)	391.690(30.734)
Hard + Soft $\lambda$ =0.5	-0.097(0.141)	15.088(0.689)	0.246(0.074)	351.066(44.974)
Adapter (Desc: 29.17M, Rec: 10.00M)	-0.103(0.194)	14.797(1.517)	0.304(0.090)	352.751(62.021)
Hard + Soft $\lambda$ =1.0	-0.024(0.165)	14.839(1.500)	0.322(0.117)	327.603(52.872)

Appendix

Table 38: Ablation study: Basic perceptual attribute prediction: Intensity (Keller).

	Intensity (Sagar)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Inter	-0.389(0.244)	0.455(0.034)	0.360(0.132)	0.283(0.050)
Only Soft $\lambda$ =2.0	-0.253(0.355)	0.455(0.054)	0.347(0.180)	0.256(0.072)
Molecule only	-0.317(0.102)	0.442(0.022)	0.372(0.051)	0.269(0.021)
Hard + Soft $\lambda$ =0.1	-0.112(0.065)	0.404(0.040)	0.273(0.166)	0.227(0.013)
No Orthogonal	-0.138(0.253)	0.398(0.072)	0.308(0.108)	0.232(0.052)

Appendix

Table 39: Ablation study: Basic perceptual attribute prediction: Intensity (Sagar).

	Intensity (Ravia)
Method	R 2 $\uparrow$	MAE $\downarrow$	Pearson $\uparrow$	MSE $\downarrow$
Molecule only	-0.407(0.352)	15.176(1.993)	0.307(0.024)	390.301(97.745)
Adapter (Desc: 29.17M, Rec: 10.00M)	0.069(0.123)	11.873(0.576)	0.202(0.288)	258.179(34.039)
Receptor only	0.058(0.135)	11.988(0.719)	0.347(0.134)	261.213(37.444)
Inter + Intra	0.044(0.142)	12.107(0.601)	0.462(0.007)	265.066(39.293)
Adapter (Desc: 10.00M, Rec: 10.00M)	0.079(0.074)	12.012(0.130)	0.426(0.119)	255.522(20.592)

Appendix

Table 40: Ablation study: Basic perceptual attribute prediction: Intensity (Ravia).

	GS-LF Multi-label Multi-class Classification
Method	AUC $\uparrow$	AUPRC $\uparrow$	MCC $\uparrow$
No Orthogonal	0.874(0.001)	0.344(0.007)	0.220(0.025)
Adapter (Desc: 29.17M, Rec: 10.00M)	0.875(0.003)	0.344(0.005)	0.250(0.024)
Only Hard	0.874(0.002)	0.342(0.002)	0.257(0.003)
Only Soft $\lambda$ =2.0	0.874(0.001)	0.347(0.002)	0.237(0.016)
Inter + Weak	0.875(0.001)	0.345(0.004)	0.241(0.027)

Appendix

Table 41: Ablation study: semantic description prediction on GS-LF dataset.

	Multi-label Regression (Keller)
Method	$R^{2}$ $\uparrow$	MAE $\downarrow$	Pearson $r$ $\uparrow$	MSE $\downarrow$
Receptor only	-0.093(0.060)	6.369(0.226)	0.233(0.061)	79.899(3.491)
Description only	-0.060(0.062)	6.271(0.129)	0.217(0.074)	77.697(3.140)
Hard + Soft $\lambda$ =0.5	-0.042(0.059)	6.089(0.350)	0.167(0.074)	75.739(6.335)
Molecule only	-0.178(0.060)	6.741(0.209)	0.330(0.050)	90.373(5.967)
Hard + Soft $\lambda$ =1.0	-0.034(0.038)	6.136(0.141)	0.243(0.097)	75.666(4.221)

Appendix

Table 42: Ablation study: semantic description prediction on Keller dataset.

	Multi-label Regression (Sagar)
Method	$R^{2}$ $\uparrow$	MAE $\downarrow$	Pearson $r$ $\uparrow$	MSE $\downarrow$
Inter	-0.554(0.082)	0.356(0.020)	0.023(0.152)	0.239(0.015)
Inter + Weak	-0.389(0.012)	0.343(0.001)	0.007(0.108)	0.233(0.006)
Molecule only	-0.677(0.218)	0.355(0.009)	0.116(0.042)	0.252(0.002)
Hard + Soft $\lambda$ =1.0	-0.334(0.161)	0.336(0.013)	-0.050(0.069)	0.225(0.014)
Inter + Intra	-0.364(0.158)	0.339(0.017)	0.042(0.176)	0.226(0.015)

Appendix

Table 43: Ablation study: semantic description prediction on Sagar dataset.

	Mixture Intensity
Method	$R^{2}$ $\uparrow$	MAE $\downarrow$	Pearson $r$ $\uparrow$	MSE $\downarrow$
Receptor only	0.224(0.085)	0.379(0.010)	0.582(0.012)	0.215(0.023)
Hard + Soft $\lambda$ =0.1	0.258(0.095)	0.360(0.028)	0.530(0.093)	0.205(0.026)
Adapter (Desc: 10.00M, Rec: 10.00M)	0.263(0.063)	0.369(0.023)	0.575(0.029)	0.204(0.018)
Inter + Weak	0.285(0.049)	0.362(0.006)	0.566(0.061)	0.198(0.013)
Inter + Intra	0.285(0.117)	0.365(0.037)	0.576(0.050)	0.198(0.032)

Appendix

Table 44: Ablation study: mixture prediction on intensity.

	Mixture Pleasantness
Method	$R^{2}$ $\uparrow$	MAE $\downarrow$	Pearson $r$ $\uparrow$	MSE $\downarrow$
Adapter (Desc: 10.00M, Rec: 10.00M)	0.404(0.260)	0.671(0.133)	0.781(0.068)	0.681(0.298)
Hard + Soft $\lambda$ =0.5	0.509(0.053)	0.610(0.042)	0.753(0.021)	0.561(0.060)
Molecule only	0.509(0.047)	0.614(0.023)	0.795(0.035)	0.561(0.054)
Adapter (Desc: 29.17M, Rec: 10.00M)	0.521(0.127)	0.591(0.066)	0.790(0.032)	0.547(0.145)
Hard + Soft $\lambda$ =0.1	0.529(0.151)	0.582(0.091)	0.786(0.028)	0.538(0.172)

Appendix

Table 45: Ablation study: mixture prediction on pleasantness.

Representation Learning

Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.

See all 773 papers

What Images Cannot Say: Language-Guided Olfactory Representation Learning
Jul 7, 2026Eleftherios Tsonis, Xi Wang, Vicky KalogeitonDisentangled Representation Learning Representation Learning
LIX, École Polytechnique, IP Paris, CNRS
GraphNOSE: A Graph Transformer in Olfaction
Sep 4, 2026Mrityunjay Sharma, Sarabeshwar Balaji, Valentina Parma +1Molecular Property Prediction Molecular Representation Learning
CSIR- Central Scientific Instruments Organisation, Sector 30-C, Chandigarh-160030, India · Academy of Scientific and Innovative Research (AcSIR), Ghaziabad-201002, India · Department of Higher Education,Himachal Pradesh,Shimla -171001,India +3
See & Sniff: Learning Visuo-Olfactory Representations
Jun 25, 2026Seongyu Kim, Seungwoo Lee, Hyeonggon Ryu +2Representation Learning Cross-Modal Representation Learning
Korea Advanced Institute of Science and Technology, Korea · Hankuk University of Foreign Studies, Korea · Ulsan National Institute of Science and Technology, Korea

Jul 7, 2026J/K move · Enter open · S save

What Images Cannot Say: Language-Guided Olfactory Representation Learning

Eleftherios Tsonis, Xi Wang, Vicky Kalogeiton

LIX, École Polytechnique, IP Paris, CNRS

Disentangled Representation Learning Representation Learning Cross-Modal Representation Learning Cross-Modal Retrieval

Open paper AskPDF arXiv

Images tell us what a scene looks like, but rarely what it would feel like to be there. While recent datasets pair visual scenes with electronic-nose measurements, aligning smell signals with images remains challenging because many olfactory cues arise from contextual environmental factors that are not directly visible in pixels. We introduce SCENT, a multimodal framework that uses language guidance as a semantic bridge between vision and olfaction. Our approach leverages Vision-Language Models (VLMs) to generate scene descriptors capturing objects, environmental context, and plausible ambient smell cues suggested by the visual scene. These descriptors provide semantic guidance for learning olfactory representations. We train a smell encoder that maps electronic-nose signals into a shared embedding space aligned with both visual and textual representations, and introduce a languageguided latent decomposition that separates object-specific odors from contextual environmental contributions. Experiments on the New York Smells dataset demonstrate that SCENT significantly improves crossmodal retrieval compared to vision-only baselines, achieving state-of-theart performance on smell-to-image and smell-to-text retrieval tasks. In addition, our framework produces interpretable olfactory representations that enable the disentanglement of complex smell mixtures. Our results reveal the importance of contextual semantic information for grounding olfactory perception in multimodal learning and pave the way for future research in this area.

Abstract

Representation Learning

Explore similar work

What Images Cannot Say: Language-Guided Olfactory Representation Learning

GraphNOSE: A Graph Transformer in Olfaction

See & Sniff: Learning Visuo-Olfactory Representations