DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain

Song Jin; Juntian Zhang; Xun Zhang; Zeying Tian; Fei Jiang; Guojun Yin; Wei Lin; Yong Liu

cs.CVApr 12, 2026

DiningBench: A Hierarchical Multi-view Benchmark for Perception and Reasoning in the Dietary Domain

Authors: Song Jin, Juntian Zhang, Xun Zhang, Zeying Tian, Fei Jiang, Guojun Yin, Wei Lin, Yong Liu,

Organizations: Gaoling School of Artificial Intelligence, Renmin University of China · Meituan · Wuhan University

Abstract

Recent advancements in Vision-Language Models (VLMs) have revolutionized general visual understanding. However, their application in the food domain remains constrained by benchmarks that rely on coarse-grained categories, single-view imagery, and inaccurate metadata. To bridge this gap, we introduce DiningBench, a hierarchical, multi-view benchmark designed to evaluate VLMs across three levels of cognitive complexity: Fine-Grained Classification, Nutrition Estimation, and Visual Question Answering. Unlike previous datasets, DiningBench comprises 3,021 distinct dishes with an average of 5.27 images per entry, incorporating fine-grained "hard" negatives from identical menus and rigorous, verification-based nutritional data. We conduct an extensive evaluation of 29 state-of-the-art open-source and proprietary models. Our experiments reveal that while current VLMs excel at general reasoning, they struggle significantly with fine-grained visual discrimination and precise nutritional reasoning. Furthermore, we systematically investigate the impact of multi-view inputs and Chain-of-Thought reasoning, identifying five primary failure modes. DiningBench serves as a challenging testbed to drive the next generation of food-centric VLM research. All codes are released in https://github.com/meituan/DiningBench.

Figures & tables

Figure 1: Overview of the DiningBench Framework. The benchmark evaluates VLMs across a hierarchy of cognitive complexity: (1) Identification (Fine-Grained Classification with hard negatives), (2) Quantification (Nutrition Estimation), and (3) Reasoning (Visual Question Answering). The pipeline utilizes multi-view imagery to assess fine-grained visual understanding.

Figure 2: DiningBench Data Construction Pipeline. The process is divided into two phases: (1) Base Data Construction, involving the filtration of raw user-generated content (UGC); and (2) Task Generation, utilizing AI-assisted pipelines (with Gemini-3-Pro-Preview) for hard negative mining, nutrition inference, and VQA generation, followed by rigorous human verification.

Name	Dish Count	Image Count	Avg Images Per Dish	Dish Categories
All	3021	15928	5.27	2060
Fine-Grained Classification	2884	15330	5.32	1977
Nutrition Estimation	1650	8856	5.37	1247
Visual Question Answering	804	839	1.04	696

Table 1: DiningBench Dataset Statistics. A summary of the total number of dishes, images, and categories across the three subsets: Fine-Grained Classification, Nutrition Estimation, and Visual Question Answering (VQA).

Figure 3: Distribution of Nutritional Values. Histograms illustrating the frequency distribution of Calories (kcal), Carbohydrates (g), Proteins (g), and Fats (g) across the DiningBench dataset, demonstrating a diverse range of nutritional profiles.

Country Distribution		VQA Category Distribution
Region	Count	Category	Count
Chinese	2086	Cuisine Technique	532
Western	286	Dietary Suggestion	219
Worldwide	252	Multi Image Analysis	35
Asian	187	Counterfactual Reasoning	18
Japanese	118

Table 3: A breakdown of the geographic distribution of cuisines and the categorization of VQA tasks.

Figure 4: Impact of Multi-View Inputs. Performance trends for Classification (Accuracy) and Nutrition Estimation (MAE, RMSE, MAPE) as the number of input images increases from 1 to 4. The results highlight the performance gains from view synthesis and the divergence between large-scale and smaller-scale models.

Figure 5: Impact of CoT on Nutrition Estimation. A radar chart comparing the Mean Absolute Percentage Error (MAPE) of various models with and without CoT prompting. Higher values indicate higher error rates, showing that CoT often degrades performance for regression tasks in smaller models.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 6: Sample Case: Fine-Grained Classification. An example of the classification task where the model must identify the correct dish ("Roasted Pumpkin Chicken Salad") from a list of visually and semantically similar distractors sourced from the same menu.

Figure 7: Sample Case: Nutrition Estimation. An example showing the prompt and expected JSON output for quantifying Calories, Protein, Carbohydrates, and Fat based on visual inputs.

Figure 8: Sample Case: Visual Question Answering. An example of a reasoning task where the model must determine if a dish meets specific dietary standards based on fine-grained visual cues like cheese crumbs.

Figure 9: Impact of Chain-of-Thought on Classification. A radar chart comparing the Classification Accuracy of various models with and without Chain-of-Thought (CoT) prompting, revealing that explicit reasoning steps can hinder direct visual discrimination in some models.

Figure 10: Impact of Chain-of-Thought on VQA. A radar chart comparing the VQA Accuracy of various models with and without Chain-of-Thought (CoT) prompting, showing mixed effectiveness depending on the model scale and capability.

Please strictly follow the Chain of Thought steps below for the analysis:

1. Visual Observation : Describe the food’s color, shape, texture, plating characteristics, and inferred cooking state.

2. Ingredient Breakdown : Identify and list the main ingredients, auxiliary ingredients, seasonings, and sauce components in detail.

3. Comprehensive Answer : Answer the question based on the original image content and the observations and breakdown above.

Appendix

Table 5: Prompt Template for Chain-of-Thought (CoT). The structured prompt used to guide models in generating step-by-step reasoning, covering visual observation, ingredient breakdown, and comprehensive answering.

You are an expert in nutrition and health science. Your task is to estimate the calories, macronutrients, health metrics, and trace element content of food based on the provided information.

In addition to the text information in food_info , you will be provided with the merchant’s promotional image ( food_picture ) and a user-uploaded image ( user_picture ). These may contain dish details, ingredients, cooking methods, or specific calorie counts. You must synthesize all available information.

Pay close attention to the following fields and logic:

1. food_unit : Indicates the measurement unit.

2. food_description : The merchant’s description. It may contain reference calorie info, but you must adhere to the following rules to obtain real, objective, and precise values:

Appendix

Table 6: Prompt for Nutrition Estimation. The detailed instruction set provided to models for estimating nutritional content, including rules for handling metadata, unit conversion, and visual inference.

# Role

You are a “Food Knowledge Graph Expert” proficient in standard naming conventions for Chinese and Western cuisine, as well as a “Visual Dataset Construction Expert.”

# Goal

Your task is to perform name standardization on the [Target Food Item] and [Initial Distractor Candidates], and ultimately output exactly 7 of the most visually deceptive distractors.

# Inputs

Appendix

Table 7: Prompt for Distractor Generation. The prompt used to standardize food names and generate “hard” negative candidates from the same menu for the Fine-Grained Classification task.

# Role Definition

You are a world-class Food Science and Computer Vision Benchmark Architect . Your expertise lies in designing highly challenging Multimodal VQA tasks to evaluate AI models on fine-grained visual recognition, nutritional reasoning, and logical inference.

# Goal

Based on the provided 4 reference images (Visual Input) and [God-view Metadata] , construct 1 high-difficulty VQA test sample.

# Input Data

Appendix

Table 8: Prompt for VQA Generation (Multi Image Analysis). The instruction used to generate high-difficulty VQA samples that focus on identifying discrepancies between promotional reference images and real user-uploaded photos.

# Role Definition

You are a “Senior Food Critic” with Michelin-star level appreciation capabilities. Your core competency lies in combining fine-grained Visual Cues with a deep Culinary Knowledge Graph to perform logically rigorous provenance reasoning.

# Task Objective

Based on the provided Food Image Metadata , construct 5-8 High-Difficulty VQA data samples.

Note: The output must be in strict JSON Array format.

Appendix

Table 9: Prompt for Culinary VQA Generation. The prompt designed for constructing complex reasoning questions related to cuisine technique, dietary suggestion, and counterfactual reasoning.

Role Definition

You are a senior Fine-grained Visual Recognition Data QA Specialist . You need to audit the validity of a “Food Recognition” multiple-choice question.

This question aims to test the model’s ability to distinguish visually similar dishes; therefore, distractors are allowed to be highly visually similar to the correct answer.

[Question Data]

1. Candidate Options:

{option_str}

Appendix

Table 10: Quality Assurance Prompt for Classification. The criteria and instructions used by AI auditors to validate the quality, difficulty, and uniqueness of Fine-Grained Classification samples.

Role Definition

You are a Food Nutrition Data Final Audit Expert with “Absolute Zero Tolerance” standards. Your task is to clean the “Food Nutrition Estimation” Golden Test Set.

Any data with flaws, ambiguity, logical conflicts, or estimation bias must be ruthlessly rejected. Your goal is to ensure the remaining data is 100% perfect and accurate.

[Data to Audit]

Standard Answer (Ground Truth):

{cal_response2_str}

Appendix

Table 11: Quality Assurance Prompt for Nutrition Data. The rigorous audit protocol for validating nutrition samples, ensuring image quality, mathematical consistency (Atwater system), and visual-data matching.

# Role

You are a Chief VQA Auditor . Your task is to build a “High-Difficulty” test set for SOTA models.

Your core principles are: Atomic, High-Difficulty, Zero-Tolerance . Any mediocre, simple, structurally chaotic, or verbose data must be immediately “sentenced to death”.

# Input Data

- Visual Context: [Model analysis based on image]

- Question: {question}

Appendix

Table 12: Quality Assurance Prompt for VQA. The strict audit guidelines used to filter VQA samples, rejecting trivial, multi-part, or hallucinated questions to ensure high benchmark difficulty.

VLM Evaluation

Vision-language model evaluation measures multimodal systems’ perception, reasoning, grounding, robustness, and reliability across controlled and real-world tasks.

See all 989 papers

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning
Jun 3, 2026Yu Zhu, Yongkang Li, Wenjie Zhu +5Vision-Language Models VLM Reasoning
Huazhong University of Science and Technology, Wuhan, China
OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice
Jul 9, 2026Qian Jiang, Zhecheng Shi, Jingpu Yang +2VLM Evaluation Medical VLMs
Northeastern University at Qinhuangdao · The Hong Kong University of Science and Technology (Guangzhou) · BeiHang University +1
OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet
Aug 4, 2026Dimitrios I. Zaridis, Traianos Tsiokris, Vasileios C. Pezoulas +6Large Vision-Language Models Fine-Grained Image Classification
Unit of Medical Technology & Intelligent Information Systems University of Ioannina Ioannina, Greece · Department of Medical Physics, School of Medicine University of Patras Patras, Greece · Computational BioMedicine Laboratory Foundation for Research and Technology Hellas Heraklion, Greece +2

Jun 3, 2026J/K move · Enter open · S save

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

Yu Zhu, Yongkang Li, Wenjie Zhu, Haoyi Jiang+4

Huazhong University of Science and Technology, Wuhan, China

Vision-Language Models VLM Reasoning

Open paper AskPDF arXiv

Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.

Abstract

VLM Evaluation

Explore similar work

Food-R1: A Unified Multi-Task Food Vision-Language Model with Reinforcement Learning

OmniFood-Bench: Evaluating VLMs for Nutrient Reasoning and Personalized Health Advice

OliveGemma: A 3 Billion Visual Language Model for Recognising the Mediterranean & European Diet