Current stereo vision-based approaches for food portion estimation are hindered by the user burden of using a mobile phone to capture multiple images or placing a reference object in the scene for scale calibration. As a step towards passive dietary intake monitoring, we introduce a method that uses stereo video from a wearable egocentric device to reconstruct food items in true physical scale to accurately estimate volume. By utilizing FoundationStereo for zero-shot stereo matching, SAM 2 for food segmentation, and TSDF fusion over a set of keyframes, our method yields a visually and metrically accurate 3D mesh of any plated food item from a single stereo recording. We leverage the unique offerings of wearables, such as a factory-calibrated stereo baseline and camera poses estimated on-device via visual-inertial odometry (VIO), to perform fully automatic reconstruction without a scale reference object in the scene. We show strong initial volume estimation results across five recorded food items, achieving a mean absolute percentage error of 9.1%.
Figures & tables
Figure 1: Our method builds a metrically accurate watertight mesh of a plated food item from two full arcs of head motion around the plate. The true physical scale of the item is recovered entirely from the stereo cameras’ calibration and on-device pose estimation. No reference object or user-provided dimension is used at any stage.
Figure 2: Method overview. (a) 40 rectified stereo keyframes spaced evenly across the device trajectory are used as inputs to (b) FoundationStereo. Metric depth maps are computed from the generated dense disparity maps using the stereo baseline and camera intrinsics. (c) Food and plate masks are generated using SAM 2 by leveraging the wearer’s gaze on the food in the first keyframe. (d) The depth maps and food masks yield a mesh using traditional TSDF fusion, whose contact region with the plate is unobserved in the camera views, guaranteeing a hole; (e) this hole is filled using the plate plane fitted in the previous stage by clipping the mesh at the plane’s surface. All holes are filled via fan triangulation, and the final watertight mesh is integrated to estimate the food’s volume.
Item
Ground Truth (mL)
Estimated (mL)
Error (%)
Apple
177.0
156.2
− 11.8
Honey Bun
219.0
190.9
− 12.8
Pizza Slice
141.0
125.2
− 11.2
Sandwich
189.0
196.6
+ 4.0
Sub Sandwich
376.0
397.3
+ 5.7
MAPE
9.1%
Table 1: Quantitative results of volume estimation against ground truth.
Figure 3: Qualitative results on food reconstruction. Our method leverages TSDF fusion to produce meshes that capture fine geometric details, such as the cavity on the apple (first column) and the irregular toppings on the pizza slice (third column).
Figure 4: Effect of replacing stereo with monocular depth estimation. Point clouds of the apple captured from three different keyframes (separated by color) produced by backprojecting 2D pixels into the shared 3D world frame using the same poses. The point clouds produced using the FoundationStereo-based depth maps intersect and form a coherent shell of an apple; those produced using the monocular model’s depth maps do not intersect, resulting in failed TSDF fusion.
Can a visually plausible food mesh be trusted to estimate the volume of consumed food? \method investigates this question using selected paired before- and after-consumption states from the MetaFood CVPR 2026 Continuous 3D Reconstruction While Eating Challenge. The submitted workflow follows a curated reconstruction protocol: SAM3 segments the food and plate regions; Hunyuan3D/SAM3D generates a dimensionless food mesh; the plate diameter provides the metric scale; the plate geometry is removed in Blender; and the remaining mesh is hole-filled, made watertight, and integrated to estimate volume. MoGe-2 is used only as an auxiliary cue for initial dish-diameter estimation when direct plate measurement is uncertain; it is not the primary scale source for the reported challenge result. \method ranks first, with an average Chamfer distance of 8.31 across 34 meshes using rigid ICP without scale correction. On 17 before- and after-pairs, it achieves 33.87% state-level volume MAPE and zero monotonicity violations, while consumed-volume MAPE remains 53.74%. The results show that surface reconstruction, metric scale, controlled mesh cleanup, watertight volume integration, and physical depletion consistency should be evaluated separately for dietary assessment. Source code and evaluation scripts will be available at github.com/GCVCG/PerBite-CVPR-MetaFood-2026.
Ahmad AlMughrabi, Farid Al-Areqi, David Fernández Gómez +4
1Universitat de Barcelona, Spain. · 4LogMeal, AIGecko Technologies S.L. · 2Grup de Tecnologies Interactives, Universitat Pompeu Fabra, Spain. +1
Accurate volume estimation of objects from visual data is a long-standing challenge in computer vision with significant applications in robotics, logistics, and smart health. Existing methods often rely on complex 3D reconstruction pipelines or struggle with the ambiguity inherent in single-view images. To address these limitations, we introduce a new method that fuses implicit 3D cues from stereo vision with explicit prior knowledge from natural language text. Our approach extracts deep features from a stereo image pair and a descriptive text prompt that contains the object's class and an approximate volume, then integrates them using a simple yet effective projection layer into a unified, multi-modal representation for regression. We conduct extensive experiments on public datasets demonstrating that our text-guided approach significantly outperforms vision-only baselines. Our findings show that leveraging even simple textual priors can effectively guide the volume estimation task, paving the way for more context-aware visual measurement systems. Code: https://gitlab.com/viper-purdue/stereo-typical-estimator.
Gautham Vinod, Bruce Coburn, Siddeshwar Raghavan +1
Purdue University, West Lafayette, Indiana, U.S.A.
Human dexterity is guided by two eyes watching two hands: binocular vision supplies the metric 3D structure that fine-grained manipulation consumes. Egocentric stereo is therefore the natural perceptual interface for robots, AR, and VR-yet metric 3D hand reconstruction from this very signal still has neither an end-to-end model nor an in-the-wild benchmark. We propose ESTHER, a model whose stereo geometry, temporal reasoning, and output representation are designed for wearable egocentric stereo. It is trained on pseudo-labels from a calibrated labeling pipeline and in turn assembles our benchmark ESTHER3D, an egocentric stereo hand dataset pairing a large in-the-wild training set of model-generated labels with a motion capture test set of true metric ground truth. Experiments show state-of-the-art accu?racy, superior external generalization, and robustness to the missing views, dropped frames, and lighting and motion blur extremes of real egocentric capture that break existing meth?ods. This robustness runs deeper than graceful degradation: stereo guidance teaches the model to bind apparent hand scale to metric depth, so it not only adapts to different stereo rigs and modalities with minimal fine-tuning, but more strikingly preserves true metric scale even after collapsing to a single monocular view.