Beyond Fine-Tuning: Robust Food Entity Linking under Ontology Drift with FoodOntoRAG
Authors: Jan Drole, Ana Gjorgjevikj, Barbara Korouši'c Seljak, Tome Eftimov
Organizations: Computer Systems Department, Jožef Stefan International Postgraduate School, Ljubljana, Slovenia · Computer Systems Department, Jožef Stefan Institute, Ljubljana, Slovenia
Standardizing food terms from product labels and menus into ontology concepts is a prerequisite for trustworthy dietary assessment and safety reporting. The dominant approach to Named Entity Linking (NEL) in the food and nutrition domains fine-tunes Large Language Models (LLMs) on task-specific corpora. Although effective, fine-tuning incurs substantial computational cost, ties models to a particular ontology snapshot (i.e., version), and degrades under ontology drift. This paper presents FoodOntoRAG, a model- and ontology-agnostic pipeline that performs few-shot NEL by retrieving candidate entities from domain ontologies and conditioning an LLM on structured evidence (food labels, synonyms, definitions, and relations). A hybrid lexical--semantic retriever enumerates candidates; a selector agent chooses a best match with rationale; a separate scorer agent calibrates confidence; and, when confidence falls below a threshold, a synonym generator agent proposes reformulations to re-enter the loop. The pipeline approaches state-of-the-art accuracy while revealing gaps and inconsistencies in existing annotations. The design avoids fine-tuning, improves robustness to ontology evolution, and yields interpretable decisions through grounded justifications.
Figures & tables
Figure 1: Visualization of the approach to named entity linking, composed of four different agents.
τ
Acc@1 (overall)
Acc@1 first
Acc@1 final
Retry rate
Syn. rate
0.4
59.8%
59.7%
59.2%
7.5%
3.8%
0.5
58.8%
59.9%
58.1%
9.3%
3.7%
0.6
57.5%
60.0%
58.4%
8.4%
4.1%
0.7
58.6%
60.2%
58.6%
9.9%
4.3%
0.8
57.2%
59.6%
57.2%
11.7%
5.8%
Table 1: Acc@1 versus confidence threshold τ on CafeteriaFCD ( n=948 ) with Gemini 2.5 Flash Lite; Klex=15 , Ksem=15 , concatenation fusion. “First” is the initial attempt; “Final” is after any retry/synonyms if triggered.
Identifying recipe entities and linking them to ontology concepts is fundamental for semantic recipe understanding. However, food mentions in recipes are often ambiguous, context-dependent, and lexically diverse, making ontology entity linking (EL) challenging. We propose SCODEL, a streamlined, model-agnostic approach based on a central idea: contextual food mentions and ontology concepts that refer to the same entity should learn the same canonical ontology-oriented definition. SCODEL uses these shared definitions as a semantic normalisation layer, providing complementary semantic information to existing EL models without requiring a new linking architecture. We further introduce ReciLinkD, the first benchmark for contextual food ontology entity linking in recipes. Experiments show that SCODEL consistently improves three modular EL models (ReFinED, GSM-EL, and BLINK): SCODEL x BLINK achieves the highest Hit@1 on seen concepts (88.33%), while SCODEL x GSM-EL achieves the highest Hit@1 on unseen concepts (83.15%), outperforming the strongest large language model (LLM) based EL baseline in each setting. Experiments on a biomedical ontology benchmark further demonstrate that SCODEL generalises beyond the food domain. Finally, we use SCODEL to ground food mentions at scale and construct an ontology-grounded recipe knowledge graph, demonstrating its utility for ontology-aware recipe retrieval and corpus-level food analysis.
Nuhu Ibrahim, Rishi Ravikumar, Ayuba Dauda +3
Department of Computer Science, The University of Manchester, United Kingdom
Recent studies have explored Vision-Language Models (VLMs) for food analysis. However, most existing methods rely primarily on supervised fine-tuning (SFT), which often limits reasoning and generalization capabilities. Moreover, high-quality large-scale nutritional annotations remain scarce. To address these issues, we introduce CalorieBench-80K, a large-scale benchmark with curated calorie labels and dietary advice annotations. To the best of our knowledge, it is the first food image benchmark to incorporate Chain-of-Thought (CoT) annotations for calorie reasoning. We also propose Food-R1, a unified food VLM trained in a multi-task learning paradigm to equip the model with broad capabilities. Food-R1 undergoes CoT-based cold-start instruction tuning, followed by reinforcement fine-tuning (RFT) using Group Relative Policy Optimization (GRPO) to improve reasoning and performance. Experiments on CalorieBench-80K and representative benchmarks show that Food-R1 consistently outperforms strong baselines across food-related tasks. The code, model weights, and benchmark annotations are available at the project repository.
Yu Zhu, Yongkang Li, Wenjie Zhu +5
Huazhong University of Science and Technology, Wuhan, China
Entity linking is a key component of many downstream NLP systems, yet existing approaches are often tied to the specific target knowledge bases and domains, limiting their real world application. In this paper, we extend LELA, a modular and domain-agnostic LLM-based entity disambiguation method, into a practical Python library that integrates zero-shot Named Entity Recognition (NER) -thereby providing a complete end-toend pipeline for entity-linking in real-world usage. We provide experimental results validating LELA's performance and robustness across diverse entity linking settings. In our demo, users can play with the system on their own input texts.
Samy Haffoudhi, Nikola Dobričić, Fabian Suchanek +1
IP Paris, LTCI, DIG · T´el´ecom Paris, Institut Polytechnique de Paris · IP Paris +1