RecipeNet: A Hierarchical Transformer for Recipe Data

Pin-Yen Huang; Sachin Chhabra; Prasanth Sai Gouripeddi; Abhinav Kumar; Baoxin Li

cs.LGAug 14, 2026

RecipeNet: A Hierarchical Transformer for Recipe Data

Authors: Pin-Yen Huang, Sachin Chhabra, Prasanth Sai Gouripeddi, Abhinav Kumar, Baoxin Li

Organizations: Arizona State University Tempe, Arizona, USA · Applied Materials Santa Clara, California, USA · University of Illinois Chicago Chicago, Illinois, USA

Abstract

Recipe data arises in domains such as materials synthesis, pharmaceutical formulation, and industrial manufacturing, where procedures are represented as ordered sequences of steps containing heterogeneous structured fields. Existing tabular learning methods typically flatten this structure into fixed-schema representations, limiting their ability to capture hierarchical field interactions and procedural dependencies. We propose RecipeNet, a hierarchical Transformer architecture that encodes field-level interactions within each step and sequential dependencies across steps through stacked Transformer encoders. Experiments on multiple recipe datasets and tasks demonstrate that RecipeNet consistently outperforms existing tabular models, highlighting the value of hierarchical and sequential modeling for recipe representation learning.

Figures & tables

Attribute	Step 1: Mixing	Step 2: Heating	Step 3: Cooling
Temperature	25 ∘ C	80 ∘ C	30 ∘ C
Pressure	–	2.5 bar	1.0 bar
Duration	5 min	20 min	10 min
Material A	20 g	–	–
Material B	10 g	–	–

Table 1. Example of a manufacturing recipe sample.

Figure 1. Overview of the proposed RecipeNet architecture.

	Solid-state reactions		Sol-gel precursor synthesis		Solution synthesis
Method	Next-step Prediction	Masked-step Prediction	Next-step Prediction	Masked-step Prediction	Next-step Prediction	Masked-step Prediction
XGBoost	0.418 $\pm$ 0.001	0.974 $\pm$ 0.000	0.359 $\pm$ 0.003	0.911 $\pm$ 0.003	0.573 $\pm$ 0.001	0.882 $\pm$ 0.001
CatBoost	0.394 $\pm$ 0.002	0.993 $\pm$ 0.001	0.262 $\pm$ 0.003	0.981 $\pm$ 0.004	0.550 $\pm$ 0.001	0.980 $\pm$ 0.001
TabNet	0.367 $\pm$ 0.008	0.633 $\pm$ 0.004	0.363 $\pm$ 0.008	0.758 $\pm$ 0.016	0.501 $\pm$ 0.005	0.739 $\pm$ 0.007
NODE	0.167 $\pm$ 0.000	0.167 $\pm$ 0.000	0.167 $\pm$ 0.000	0.167 $\pm$ 0.000	0.167 $\pm$ 0.000	0.167 $\pm$ 0.000
Transformer	0.439 $\pm$ 0.010	0.752 $\pm$ 0.033	0.331 $\pm$ 0.026	0.939 $\pm$ 0.011	0.546 $\pm$ 0.004	0.787 $\pm$ 0.005

Table 2. Performance comparison of tabular learning models across recipe datasets and downstream tasks.

Figure 2. Visualization of learned recipe representations using t-SNE. Different colors correspond to different target classes.

Figure 3. Comparison of total training time between RecipeNet and other neural tabular learning models.

Component	Next-step Prediction	Masked-step Prediction
RecipeNet (full model)	0.406	0.999
w/o Hierarchy	0.398	0.995
w/o Step Encoder	0.334	0.973
w/o Recipe Encoder	0.351	0.994
w/o Step Embedding	0.369	0.997
w/o Field Embedding	0.391	0.994

Table 3. Ablation study evaluating the contribution of each component in RecipeNet.

Representation Learning

Representation learning develops data transformations that expose useful structure for prediction, transfer, generation, and control.

See all 804 papers

ReGraph: Learning to Generate Recipe Graphs from Food Images
Aug 7, 2026Guoshan Liu, Bin Zhu, Pengkun Jiao +3Graph Generation Graph Representation Learning
Institute of Trustworthy Embodied AI, Fudan University, China and Shanghai Key Laboratory of Multimodal Embodied AI, China · Singapore Management University, Singapore
A framework for recipe data structure with applications for culinary and nutritional insights
Aug 12, 2026Mansi Goel, Sumit Bhagat, Saloni Srivastava +3Knowledge Representation Document Information Extraction
Department of Computational Biology, Indraprastha Institute of Information Technology Delhi (IIIT-Delhi), New Delhi, 110020, India · Infosys Centre for Artificial Intelligence, Indraprastha Institute of Information Technology Delhi (IIIT-Delhi), New Delhi, 110020, India · Center of Excellence in Healthcare, Indraprastha Institute of Information Technology Delhi (IIIT-Delhi), New Delhi, 110020, India +2
H2Table: Hierarchical Hypergraph-Enhanced Large Language Models for Complex Table Reasoning
Sep 1, 2026Jia Ling, Yangfan Wang, Chen Tang +4Table QA Hierarchical Representation Learning
Harbin Institute of Technology · AI Research Center, Midea Group (Shanghai) Co., Ltd. · Changchun University of Science and Technology +1

Aug 7, 2026J/K move · Enter open · S save

ReGraph: Learning to Generate Recipe Graphs from Food Images

Guoshan Liu, Bin Zhu, Pengkun Jiao, Jingjing Chen+2

Institute of Trustworthy Embodied AI, Fudan University, China and Shanghai Key Laboratory of Multimodal Embodied AI, China · Singapore Management University, Singapore

Graph Generation Graph Representation Learning Multimodal Large Language Models

Open paper AskPDF arXiv

Recent Large Multimodal Models (LMMs) have achieved impressive performance in recipe generation from food images.However, cooking is a structured transformation process in which ingredients undergo state changes through ordered actions,while free-form recipe language leaves the corresponding entities, intermediate states, and dependencies largely implicit and entangled.A graph representation makes this procedural knowledge explicit and compositional, providing a structured basis for assessing whether model outputs encode process-level knowledge rather than merely presenting plausible textual descriptions. To address this limitation, we present ReGraph, a large-scale recipe graph dataset that represents ingredients, cooking actions, and tools as entities, uses entity attributes to describe ingredient state changes, and employs typed relations to encode manipulation targets, destinations, and procedural ordering. ReGraph further incorporates explicit Recipe Reasoning Chain-of-Thought (RR-CoT) traces, providing auxiliary supervision for procedural decomposition and structured graph generation. Building on ReGraph, we propose Recipe Graph Learning (RGL), a two-stage framework that enables LMMs to generate a plausible fine-grained cooking workflow from a food image in the form of a structured recipe graph. Under a deterministic, schema-aware matching protocol, our experiments reveal a substantial gap between text-generation quality and recoverable procedural structure: recipes produced by existing approaches achieve competitive text-generation scores yet yield limited reference-aligned entity and relation structure under the ReGraph schema. In contrast, across two representative LMM backbones, RGL consistently improves the generation of cooking entities and procedural relations, while our analysis further shows that fine-grained ingredient-state capture remains the most challenging dimension.

Abstract

Representation Learning

Explore similar work

ReGraph: Learning to Generate Recipe Graphs from Food Images

A framework for recipe data structure with applications for culinary and nutritional insights

H2Table: Hierarchical Hypergraph-Enhanced Large Language Models for Complex Table Reasoning