cs.CVOct 8, 2026

Reference-Free Food Volume Estimation from Egocentric Stereo Video

Authors: Vivan Kumar, Gautham Vinod, Fengqing Zhu

Organizations: Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Current stereo vision-based approaches for food portion estimation are hindered by the user burden of using a mobile phone to capture multiple images or placing a reference object in the scene for scale calibration. As a step towards passive dietary intake monitoring, we introduce a method that uses stereo video from a wearable egocentric device to reconstruct food items in true physical scale to accurately estimate volume. By utilizing FoundationStereo for zero-shot stereo matching, SAM 2 for food segmentation, and TSDF fusion over a set of keyframes, our method yields a visually and metrically accurate 3D mesh of any plated food item from a single stereo recording. We leverage the unique offerings of wearables, such as a factory-calibrated stereo baseline and camera poses estimated on-device via visual-inertial odometry (VIO), to perform fully automatic reconstruction without a scale reference object in the scene. We show strong initial volume estimation results across five recorded food items, achieving a mean absolute percentage error of 9.1%.

Figures & tables

Figure 1: Our method builds a metrically accurate watertight mesh of a plated food item from two full arcs of head motion around the plate. The true physical scale of the item is recovered entirely from the stereo cameras’ calibration and on-device pose estimation. No reference object or user-provided dimension is used at any stage.
Figure 2: Method overview. (a) 40 rectified stereo keyframes spaced evenly across the device trajectory are used as inputs to (b) FoundationStereo. Metric depth maps are computed from the generated dense disparity maps using the stereo baseline and camera intrinsics. (c) Food and plate masks are generated using SAM 2 by leveraging the wearer’s gaze on the food in the first keyframe. (d) The depth maps and food masks yield a mesh using traditional TSDF fusion, whose contact region with the plate is unobserved in the camera views, guaranteeing a hole; (e) this hole is filled using the plate plane fitted in the previous stage by clipping the mesh at the plane’s surface. All holes are filled via fan triangulation, and the final watertight mesh is integrated to estimate the food’s volume.
ItemGround Truth (mL)Estimated (mL)Error (%)
Apple177.0156.2−- 11.8
Honey Bun219.0190.9−- 12.8
Pizza Slice141.0125.2−- 11.2
Sandwich189.0196.6++ 4.0
Sub Sandwich376.0397.3++ 5.7
MAPE9.1%
Table 1: Quantitative results of volume estimation against ground truth.
Figure 3: Qualitative results on food reconstruction. Our method leverages TSDF fusion to produce meshes that capture fine geometric details, such as the cavity on the apple (first column) and the irregular toppings on the pizza slice (third column).
Figure 4: Effect of replacing stereo with monocular depth estimation. Point clouds of the apple captured from three different keyframes (separated by color) produced by backprojecting 2D pixels into the shared 3D world frame using the same poses. The point clouds produced using the FoundationStereo-based depth maps intersect and form a coherent shell of an apple; those produced using the monocular model’s depth maps do not intersect, resulting in failed TSDF fusion.
More in this topic

3D Reconstruction

3D reconstruction recovers a scene’s shape, appearance, and camera motion from images, video, depth, or other sensor measurements.

See all 950 papers

Explore similar work

CardsList

  1. PerBite: A Curated Diagnostic Workflow for Bite-Aware Food Volume Estimation

    Jun 1, 2026Ahmad AlMughrabi, Farid Al-Areqi, David Fernández Gómez +43D Reconstruction3D Mesh Reconstruction

    1Universitat de Barcelona, Spain. · 4LogMeal, AIGecko Technologies S.L. · 2Grup de Tecnologies Interactives, Universitat Pompeu Fabra, Spain. +1

  2. Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception

    Apr 10, 2026Gautham Vinod, Bruce Coburn, Siddeshwar Raghavan +1

    Purdue University, West Lafayette, Indiana, U.S.A.

  3. ESTHER: Egocentric Stereo Hand Estimation and Reconstruction in the Wild

    Sep 28, 2026Hongyu Ma, Hairong Qu, Shiqi Zhao +2Articulated Object Reconstruction3D Hand Pose Estimation

    City University of Hong Kong