cs.AIOct 1, 2026

TasteBench: Multimodal Benchmark for Sensory Prediction, from Molecules to Sustainable Foods

Authors: Anna T. Thomas, Sohum Patnaik, Caroline Cotto, Benjamin Sanchez-Lengeling

Organizations: Stanford Computer Science · Food Intelligence Lab · NECTAR · University of Toronto Chemical Engineering

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Sustainable protein discovery lacks the fast computational proxies, analogous to molecular docking or density functional theory, that accelerate drug and materials discovery. Evaluating whether a novel food tastes like its animal-based target requires expensive human sensory panels, bottlenecking the design-build-test loop. We introduce TasteBench, a multimodal benchmark and privacy-preserving competition for sensory prediction, spanning two tasks: a food-level ranking task built on 21K+ human evaluations across 215 plant-based foods in 24 product categories, yielding 935 within-category ranking pairs, and a supporting molecular-level taste classification task over 15K flavor molecules. To enable rigorous interpretation of model performance, we characterize the ground truth: inter-rater agreement among panelists is low (Krippendorff's α=.077α= .077), and the split-half reliability ceiling of panel-aggregated rankings is .825, establishing the range within which ML systems on this benchmark should be assessed. We evaluate baselines across four input modalities; on the same pairs panelists rated, the best model achieves .661 pairwise accuracy, competitive with the median individual panelist (.650), and .683 across all within-category pairs. TasteBench provides the evaluation infrastructure and baselines for measuring progress on computational screening for sustainable protein discovery.

Figures & tables

Figure 1: Overview of TasteBench , a multimodal benchmark built on 21k+ human evaluations across 215 plant-based foods in 24 product categories, and 15K flavor molecules.
Figure 2: Models perform competitively with small human panels . Pairwise accuracy vs. panel size (left) and discrete comparison (right, Table 1 ).
ModelPw. Acc.ρS\rho_{\mathrm{S}}τK\tau_{\mathrm{K}}R@1R@2R@3
Gemini 3.1 Pro.654 † [.619,.713].370 † [.250,.509].298 † [.179,.409].271 † [.167,.424].514 [.476,.701].578 † [.491,.786]
Qwen 3.5 397B.630 † [.591,.688].320 † [.203,.459].255 † [.141,.365].229 † [.132,.354].354 † [.215,.483].500 † [.375,.656]
MMRF (cosine).574 [.528,.635].212 † [.077,.380].151 † [.023,.278].208 † [.104,.323].292 [.135,.396].333 [.181,.379]
MMRF (L2).571 [.524,.632].202 † [.064,.358].153 † [.031,.277].222 † [.104,.335].403 † [.285,.576].542 † [.438,.729]
Ridge.579 [.534,.632].160 [.028,.302].115 † [ −.005-.005 ,.226].250 † [.181,.435].292 [.118,.396].542 † [.493,.749]
Bradley–Terry.610 † [.571,.661].266 † [.156,.402].178 † [.064,.287].250 † [.181,.422].292 [.125,.396].375 [.187,.458]
Table 2: Ensembles of supervised models and LLMs outperform the median human panelist. Ranking metrics on 215 NECTAR plant-based products (935 within-category pairs; LOOCV) with 95% BCa CIs. Bold = best in column; † = CI overlaps the leader (no significant difference at 95%)
ModelAccuracyPrecisionRecallF1AUROC
Reported by Zimmermann et al. (2025)
XGBoost (fp).899.817.740.766.852
XGBoost (fp+desc).896.884.740.778.851
Balanced RF (fp).797.585.732.601.839
FART.894 † [.880,.906].879 [.675,.897].739 † [.668,.869].774 † [.671,.875].974 † [.968,.979]
This work
Table 3: Taste classification on the FART test split (2,254 molecules). Baselines from Zimmermann et al. (2025) ; 95% BCa CIs (10,000 resamples). Bold = best; † = CI overlaps leader. Per-class breakdown in Table 13 .
ModelC == FARTC == GNNC == ChemBERTa-2ΔGNN\Delta_{\text{GNN}}ΔChem\Delta_{\text{Chem}}
Unsupervised
MMRF (cosine).574 [.528,.635].566 [.518,.624].573 [.526,.633]−- .008−- .002
MMRF (L2).571 [.524,.632].563 [.518,.619].568 [.522,.628]−- .008−- .002
Supervised – linear
Ridge.579 [.534,.632].561 [.513,.614].598 [.554,.655]−- .018++ .019
Supervised – pairwise
Table 4: Molecular-encoder quality does not transfer to product-level ranking. Pairwise accuracy (935 within-category pairs, LOOCV), compound block swapped, all else fixed. 95% BCa CIs (10,000 resamples). Δ\Delta relative to FART; all lie inside their CIs.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Hierarchy of cost in sustainable protein evaluation.
Figure 4: Prompt template for LLM-based models. Feature ablation removes the corresponding XML blocks.
GroupFeatures
Meat (14 cat.)Total Fat, Sodium, Protein, Dietary Fiber
Non-Sweet Dairy (6 cat.)Total Fat, Sodium
Cheese (2 cat.)Total Fat, Sodium, Total Carbs
Sweet Dairy (2 cat.)Total Fat, Total Sugars
Appendix
Table 5: Nutrition features by category group. All values normalized to per-100g serving.
ModelMicroMacroΔ\DeltaPartial pooling (EB)
Ensemble (BT ++ Gemini)
NNLS.683.641−- .042.682
Rank average.671.629−- .041.668
Mean.623.595−- .028.613
Zero-shot LLMs
Gemini 3.1 Pro.654.633−- .021.666
Appendix
Table 14: Category-balanced evaluation preserves the ranking and the best model. Pairwise accuracy on the 935 within-category pairs under three aggregation schemes. Micro is the paper’s reported metric (Table 2 ); macro weights each of the 24 categories equally; EB is DerSimonian–Laird partial pooling toward the precision-weighted random-effects grand mean, with each category’s sampling variance taken from its product count. Δ\Delta == macro −- micro. Point estimates only: intervals for the macro and EB columns come from a within-category percentile bootstrap rather than the BCa procedure used elsewhere in the paper, so they are not comparable to the intervals in Table 2 and are omitted here. Spearman correlation between the macro and micro orderings is .881.
Min. productsCategories keptMicroMacroRank (micro / macro)
524.683.6411 / 1
621.689.6611 / 1
720.694.6741 / 1
818.694.6721 / 1
916.701.6821 / 1
1012.700.6741 / 1
Appendix
Table 15: Dropping small categories does not change the best model. The best model (NNLS over Bradley–Terry and Gemini 3.1 Pro) as categories below each product-count threshold are excluded. Rank is among the twelve models of Table 14 , under micro and macro averaging respectively. The smallest category has 5 products, so the first row is the full benchmark.
QuantityValue
Panel
Panelist vs. another single panelist, pairwise ordering.547
Median panelist vs. panel mean, within block.650 (IQR .500–.750, n=627n=627 )
Krippendorff’s α\alpha (ordinal, per block).077 (mean)
Kendall’s WW (rank concordance, per block).108 (median)
Split-half reliability of the panel mean.825
Appendix
Table 16: The model against individual panelists, on matched pairs. Agreement quantities for the panel and for the best model. The panelist rows are computed on the within-block pairs each panelist actually rated; the model is scored on those same pairs in the first model row, and on all 935 within-category pairs in the second. The category-balanced row is the partial-pooled figure from Table 14 . Rows are not mutually comparable: the panelist median is a per-panelist quantity on within-block pairs, not a category-macro one.
SignaturePrevalenceΔ\Delta accuracy (flagged −- clean)Fisher pp
Product identification12.5%++ .042.403
Brand mention (mostly analogy)10.2%++ .013.820
External-performance recall2.4%−- .026.821
Any signature14.2%++ .032.489
Appendix
Table 17: Recognition rarely surfaces, and it does not track accuracy. Prevalence of each signature in the saved reasoning for the ingredients-plus-image configuration, and the accuracy difference between the pairs where it appears and the rest. Scored on the 931 pairs for which a graded judgment and a Bradley–Terry comparison both exist: of the 935 within-category pairs, three have tied panel means and are not gradable, and one more is unmatched. Fisher’s exact test, two-sided; no signature reaches significance. Signatures are not mutually exclusive.
SplitnnGemini −- Bradley–Terry
Recognized116++ .026
Not recognized815++ .052
Interaction −- .026, 95% CI [ −- .159, ++ .108] (n.s.)
Appendix
Table 18: The LLM’s margin is not concentrated where it recognized the product. Per-pair margin of Gemini 3.1 Pro over Bradley–Terry, split by whether the reasoning identified the product. Bradley–Terry sees only features, so it holds pair difficulty constant. Over these 931 pairs Gemini scores .662 and Bradley–Terry .613, a margin of ++ .048; both differ slightly from Table 2 , which scores the same models over all 935 pairs including the three ties. Bradley–Terry also improves on the recognized subset (.605 to .672), which is what identifies those pairs as easier rather than leaked. The interaction interval is a within-category bootstrap.
ArmOriginalBrand-blindΔ\Delta acc.Flip rateMcNemar ppBrand mentions
Gemini 3.1 Pro.639.634−- .00512.6%.71210.8% →\to 1.7%
Appendix
Table 19: Forbidding brand reasoning barely moves accuracy. Original versus brand-blind system prompt, both re-run fresh in the same session so that model drift cancels; the user prompt is byte-identical between arms, and 932 pairs are scored in both. Flip rate is the fraction of pairs whose predicted winner changes. The brand-mention rate is measured on the same reasoning scan as Table 17 .
Figure 5: Worked example of the ensemble pipeline for one plant-based burger (Product A) in the Burgers category ( n=10n=10 ). All four modalities feed the supervised Bradley–Terry scorer through encoding and fusion; the non-molecular modalities are also read in raw form by the zero-shot LLM scorer (Gemini 3.1 Pro). The two scores are combined via non-negative least squares to produce the predicted within-category ranking.
More in this topic

Benchmark Design

Benchmark design creates tasks, datasets, metrics, and protocols that make AI capabilities measurable and comparisons credible.

See all 391 papers

Explore similar work

CardsList
Apr 15, 2026

FoodSense: A Multisensory Food Dataset and Benchmark for Predicting Taste, Smell, Texture, and Sound from Images

Humans routinely infer taste, smell, texture, and even sound from food images a phenomenon well studied in cognitive science. However, prior vision language research on food has focused primarily on recognition tasks such as meal identification, ingredient detection, and nutrition estimation. Image-based prediction of multisensory experience remains largely unexplored. We introduce FoodSense, a human-annotated dataset for cross-sensory inference containing 66,842 participant-image pairs across 2,987 unique food images. Each pair includes numeric ratings (1-5) and free-text descriptors for four sensory dimensions: taste, smell, texture, and sound. To enable models to both predict and explain sensory expectations, we expand short human annotations into image-grounded reasoning traces. A large language model generates visual justifications conditioned on the image, ratings, and descriptors. Using these annotations, we train FoodSense-VL, a vision language benchmark model to produce both multisensory ratings and grounded explanations directly from food images. This work connects cognitive science findings on cross-sensory perception with modern instruction tuning for multimodal models and shows that many popular evaluation metrics are insufficient for visually sensory inference.

Sabab Ishraq, Aarushi Aarushi, Juncai Jiang +1

College of Engineering and Computer Science, University of Central Florida, Orlando, FL, USA · College of Business Administration, University of Central Florida, Orlando, FL, USA · Institute of Artificial Intelligence, University of Central Florida, Orlando, FL, USA

VLM Evaluation
AskPDF
Oct 5, 2026

TasteVal: Measuring the Experimental Research Taste of AI Systems Against Human Experts

We introduce TasteVal, a benchmark to evaluate the experimental research taste of frontier models. We define research taste as the ability to pick interesting problems to solve, design experiments, and interpret experimental results. TasteVal measures the experimental component of research taste; given a fixed research problem, we measure how well a model iteratively designs experiments and draws conclusions from their outcomes. We operationalize experimental research taste as compute efficiency; a Researcher who reaches the same score as an expert human using half the serial experimental compute has twice the experimental taste. Experimental taste thus acts as a multiplier on experimental compute, making it a key input to forecasts of AI progress. TasteVal consists of 8 novel, challenging, open-ended tasks representative of frontier AI R&D. To isolate taste from coding ability, the model under evaluation acts as a Researcher that iteratively designs experiments while a fixed Coder agent implements them and reports their results. The Researcher executes until either the 40 H100 hour or 120 wall-clock hour budgets are exhausted. We recruit 24 human experts, at least 2 per task, and take the best expert attempt per task as the expert baseline. We evaluate 20 models released between 2023 and 2026. The best-performing model, Opus 5.5, exceeds our expert baseline, with a compute multiplier of 2.3x (95% CI 1.15-4.37), at roughly 1/30 of our baseliners' average per-run cost. On TasteVal, the compute multiplier of frontier models has doubled approximately every 3.0 months since December 2025 (95% CI 1.7-5.0), up from every 14 months between 2023 and December 2025. Measured by final normalized performance, frontier models show no trend break, doubling every 14.6 months. To keep TasteVal uncontaminated, we do not release the tasks.

Oliver Jaffe, Dane Sherburn

P-Zero Research

LLM EvaluationAI Agent Benchmarks
AskPDF
May 29, 2026

FAM-Bench: Benchmarking Condition-Aware Food-as-Medicine Reasoning in Vision-Language Models

Food AI can recognize dishes, ingredients, and nutrients. However, these abilities do not show whether a dish is suitable for a person with a specific health condition. They also do not test whether such decisions are grounded in the relevant food evidence. To address this gap, we introduce FAM-Bench, a Food-as-Medicine benchmark comprising 2,500 question instances constructed from 1,811 recipes across 13 diet-related health conditions. Gold annotations are derived from rule-based dietary guidance informed by published sources and nutrition professionals. The benchmark contains two complementary tasks: dish-level suitability assessment and comparative dish analysis. We evaluate open-weight and proprietary vision-language models under four prompting modes. Results reveal a substantial gap between suitability decisions and their supporting rationales, as well as limited performance on comparative ranking. These findings highlight persistent challenges in grounded, condition-aware food reasoning.

Mingyang Mao, Bhargav Rishi Medisetti, Utkarsh Grover +4

Department of Electrical Engineering, University of South Florida · Muma College of Business, University of South Florida · Computer Science, University of Copenhagen

HealthcareClinical Decision SupportMultimodal ReasoningMultimodal Model Evaluation
AskPDF