Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception

Gautham Vinod; Bruce Coburn; Siddeshwar Raghavan; Fengqing Zhu

cs.CVApr 10, 2026

Not Your Stereo-Typical Estimator: Combining Vision and Language for Volume Perception

Authors: Gautham Vinod, Bruce Coburn, Siddeshwar Raghavan, Fengqing Zhu

Organizations: Purdue University, West Lafayette, Indiana, U.S.A.

Abstract

Accurate volume estimation of objects from visual data is a long-standing challenge in computer vision with significant applications in robotics, logistics, and smart health. Existing methods often rely on complex 3D reconstruction pipelines or struggle with the ambiguity inherent in single-view images. To address these limitations, we introduce a new method that fuses implicit 3D cues from stereo vision with explicit prior knowledge from natural language text. Our approach extracts deep features from a stereo image pair and a descriptive text prompt that contains the object's class and an approximate volume, then integrates them using a simple yet effective projection layer into a unified, multi-modal representation for regression. We conduct extensive experiments on public datasets demonstrating that our text-guided approach significantly outperforms vision-only baselines. Our findings show that leveraging even simple textual priors can effectively guide the volume estimation task, paving the way for more context-aware visual measurement systems. Code: https://gitlab.com/viper-purdue/stereo-typical-estimator.

Figures & tables

Figure 2: Overview of Our Method: Stereo images are passed through a feature extractor to generate embeddings, which are concatenated and used for classification to obtain the corresponding text prior for each class. The text embedding is generated from the constructed prompt with the text and volume prior. The combined image and text features are projected into a shared space. Finally, a regression network estimates the volume $\hat{V}_{\textit{est}}$ .

Figure 3: Visualization of our multimodal projection - The image and text features each form three separate clusters corresponding to the same class. Our projection maps these modalities into a unified space, where semantically similar image and text features cluster together.

Method	MetaFood3D Dataset					OmniObject3D Dataset
Method	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$	r $\uparrow$	$R^{2}$ $\uparrow$	$\cos(\theta)$ $\uparrow$	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$	r $\uparrow$	$R^{2}$ $\uparrow$	$\cos(\theta)$ $\uparrow$
Baseline	164.19	526.79	0.00	-0.01	0.68	140.28	444.57	0.00	-0.02	0.60
Category Mean	134.86	202.88	0.46	0.07	0.72	118.47	87.46	0.63	0.35	0.76
RGB Est. [ 36 ]	151.98	64.65	0.57	-0.37	0.80	79.53	68.34	0.81	0.46	0.85
Depth Recon. [ 8 ]	210.45	287.63	0.54	-1.08	0.78	125.34	175.97	0.46	0.21	0.61
3D Assisted [ 39 ]	134.52	97.84	0.48	-0.57	0.68	-	-	-	-	-

Table 1: Volume Estimation Comparison. We evaluate performance on the MetaFood3D and OmniObject3D datasets using five metrics: MAE, MAPE, Pearson correlation (

r

R^{2}

, and cosine similarity (

\cos(\theta)

). Our method consistently outperforms all baselines. The next-best methods are underlined . The “Baseline” refers to predicting the dataset’s mean volume for all inputs. The “Category Mean” predicts the mean volume of each sample’s category. Results from “3D Assisted” are omitted for OmniObject3D as it requires a checkerboard pattern absent in that dataset.

Stereo Methods	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$	$\mathbf{r}$ $\uparrow$	$\boldsymbol{\cos(\theta)}$ $\uparrow$
Baseline	151.85	845.69	0	0.707
Category Mean	134.86	202.88	0.46	0.72
SIFT Recon. [ 4 ]	345.34	610.92	-0.01	0.17
ORB Recon. [ 32 ]	2718.38	870.80	0.14	0.22
LightGlue Recon. [ 16 ]	203.85	221.13	0.18	0.46
NeRF Stereo [ 22 ]	104.59	132.67	0.56	0.81

Table 2: Stereo Volume Estimation Comparison. We evaluate performance on MetaFood3D using four metrics: MAE, MAPE, Pearson correlation (

r

), and cosine similarity (

\cos(\theta)

). Our method consistently outperforms all baselines. The “Baseline” refers to predicting the dataset’s mean volume for all inputs. The “Category Mean” predicts the mean volume of each sample’s category.

Figure 4: Error Distribution of Volume Estimation Methods. We highlight our method (blue) against the second-best performing method, GPT-5 with context (orange), with regards to MAPE. (Top) The Cumulative Distribution Function (CDF) of absolute errors on a log scale where a shift or curve to the upper left indicates better performance. (Bottom) The Kernel Density Estimation (KDE) of percentage errors where a tall peak near zero with significant delay indicates better performance.

Module	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$
Stereo Images Only	77.33	38.35
Text Only	188.50	100.00
Stereo + Text (Ours)	47.95	22.54

Table 3: Importance of Our Multi-Modal Approach. Using only the text features doesn’t provide the model with much information but the combination of stereo features with text outperforms each individual component.

# of Images	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$	GFLOPs $\downarrow$
1 image	71.25	35.11	0.12
2 images	58.72	31.49	0.17
5 images	62.10	31.62	0.32
10 images	63.31	30.91	0.57
20 images	53.57	29.68	1.07
50 images	51.54	26.74	2.58

Table 4: Tradeoff between number of images and performance. Our stereo based approach meant to leverage current stereo image capture devices achieves an incredible tradeoff between performance and computation.

Method	Volume MAE (mL) $\downarrow$	Energy MAE (kCal) $\downarrow$	Protein MAE (g) $\downarrow$	Carb MAE (g) $\downarrow$	Fat MAE (g) $\downarrow$
Baseline	153.89	245.88	12.01	20.45	13.29
Category Mean	120.17	138.09	5.86	16.33	5.69
3D Assisted [ 39 ]	140.01	146.76	7.65	14.26	6.75
GPT-5 (w/context) [ 42 ]	87.69	115.42	7.02	9.81	5.38
RGB Only [ 36 ]	151.98	192.90	10.43	17.44	9.24
Stereo Recon. [ 4 ]	137.14	193.95	10.84	16.16	9.78

Table 5: Dietary Assessment. Our method accurately predicts the nutritional content of the foods in the image based on our superior volume estimation by achieving the lowest MAE as compared to other established dietary assessment methods. For MFP3D [ 21 ] , only the values reported in the paper are shown.

Backbone	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$
DeiT Small	86.88	39.13
DeiT Base	67.19	34.07
ViT B/32	67.04	44.56
ViT L/14	66.95	30.75
ResNet50 - CLIP	89.70	47.84
ResNet101 - CLIP	91.47	46.97

Table 6: Comparison of feature extractors. Transformer-based models, especially the CLIP ViT-L/14@336px variant, provide the most discriminative features for volume estimation.

	MAE (mL) $\downarrow$	MAPE (%) $\downarrow$
Text prompt 1	59.56	34.98
Text prompt 2	61.98	34.05
Text prompt 3	65.81	35.26
Text prompt 4	64.27	35.54
Text prompt 5 ( ours)	61.85	30.88

Table 7: Performance of different text prompts. The results demonstrate how variations in prompt formulation affect the model’s predictive performance.

Method	MAE (mL)	MAPE (%)
Baseline	150.02	553.96
RGB Only	221.07	299.52
Ours	40.50	22.38

Table 8: Random Split Generalization. The random split into training and testing splits ensures that there are categories and items that the model has never “seen” and yet the performance is significantly better than the RGB estimation method which was also trained with the same training and testing split.

Figure 5: Error Distribution of Volume Estimation Methods. We highlight our method (blue) against the second-best performing method, GPT-5 with context (orange), with regards to MAPE. (Top) The Cumulative Distribution Function (CDF) of absolute errors on a log scale where a shift or curve to the upper left indicates better performance. (Bottom) The Kernel Density Estimation (KDE) of percentage errors where a tall peak near zero with significant delay indicates better performance.

Method	MAE (mL)	MAPE (%)
Baseline	165.75	836.50
RGB Only	174.57	212.96
GPT-5	92.38	198.83
Ours	96.08	178.86

Table 9: Cross Dataset Validation. The model tested on “out-of-distribution” images still outperforms the baseline and the RGB Only method which was trained and tested on the same data as our method.

Figure 6: Error Distribution of Volume Estimation Methods. We highlight our method (orange) against the second-best performing method, GPT-5 with context (blue), with regards to MAPE. (Top) The Cumulative Distribution Function (CDF) of absolute errors on a log scale where a shift or curve to the upper left indicates better performance. (Bottom) The Kernel Density Estimation (KDE) of percentage errors where a tall peak near zero with significant delay indicates better performance.

Figure 7: Predicted volumes plotted against ground truth volumes for MetaFood. We plot our method against the second-best performing monocular method, GPT-5 with context, and overlay a line of best fit to help visualize correlation with a “perfect estimation” line.

Figure 8: Predicted volumes plotted against ground truth volumes for OmniObject. We plot our method against the second-best performing monocular method, GPT-5 with context, and overlay a line of best fit to help visualize correlation with a “perfect estimation” line.

Reference-Free Food Volume Estimation from Egocentric Stereo Video
Oct 8, 2026Vivan Kumar, Gautham Vinod, Fengqing Zhu3D Reconstruction Stereo Vision
Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA
Lightweight Neural Framework for Robust 3D Volume and Surface Estimation from Multi-View Images
Jun 22, 2026Diego E. Farchione, Ramzi Idoughi, Peter Wonka3D Reconstruction
King Abdullah University of Science and Technology (KAUST), Saudi Arabia
Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Properties
Aug 7, 2026Ali Bahri, Hongliang Li, Soufiane Lamghari +2Object-Centric Representation Learning Materials Property Prediction
Huawei Noah’s Ark Lab, Canada · Huawei Noah’s Ark Lab, Hong Kong SAR, China

Oct 8, 2026J/K move · Enter open · S save

Reference-Free Food Volume Estimation from Egocentric Stereo Video

Vivan Kumar, Gautham Vinod, Fengqing Zhu

Elmore Family School of Electrical and Computer Engineering, Purdue University, West Lafayette, IN 47907, USA

3D Reconstruction Stereo Vision

Open paper AskPDF arXiv

Current stereo vision-based approaches for food portion estimation are hindered by the user burden of using a mobile phone to capture multiple images or placing a reference object in the scene for scale calibration. As a step towards passive dietary intake monitoring, we introduce a method that uses stereo video from a wearable egocentric device to reconstruct food items in true physical scale to accurately estimate volume. By utilizing FoundationStereo for zero-shot stereo matching, SAM 2 for food segmentation, and TSDF fusion over a set of keyframes, our method yields a visually and metrically accurate 3D mesh of any plated food item from a single stereo recording. We leverage the unique offerings of wearables, such as a factory-calibrated stereo baseline and camera poses estimated on-device via visual-inertial odometry (VIO), to perform fully automatic reconstruction without a scale reference object in the scene. We show strong initial volume estimation results across five recorded food items, achieving a mean absolute percentage error of 9.1%.

Abstract

Explore similar work

Reference-Free Food Volume Estimation from Egocentric Stereo Video

Lightweight Neural Framework for Robust 3D Volume and Surface Estimation from Multi-View Images

Vision Meets WiFi: Physics-Grounded Estimation of Volumetric Mechanical Properties