Learning to Weigh Waste: A Physics-Informed Multimodal Fusion Framework and Large-Scale Dataset for Commercial and Industrial Applications

Md. Adnanul Islam; Wasimul Karim; Md Mahbub Alam; Subhey Sadi Rahman; Md. Abdur Rahman; Arefin Ittesafun Abian; Mohaimenul Azam Khan Raiaan; Kheng Cher Yeo

cs.CVMar 1, 2026

Learning to Weigh Waste: A Physics-Informed Multimodal Fusion Framework and Large-Scale Dataset for Commercial and Industrial Applications

Authors: Md. Adnanul Islam, Wasimul Karim, Md Mahbub Alam, Subhey Sadi Rahman, Md. Abdur Rahman, Arefin Ittesafun Abian, Mohaimenul Azam Khan Raiaan, Kheng Cher Yeo,

Organizations: Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh and also with Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Department of Data Science and Artificial Intelligence, Monash University, 3800, Melbourne, Australia, and also with Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Faculty of Science and Technology, Charles Darwin University, Darwin, NT 0810, Australia · Faculty of Arts and Society, Charles Darwin University, Casuarina, Darwin, NT 0810, Australia

Abstract

Accurate weight estimation of commercial and industrial waste is important for efficient operations, yet image-based estimation remains difficult because similar-looking objects may have different densities, and the visible size changes with camera distance. Addressing this problem, we propose Multimodal Weight Predictor (MWP) framework that estimates waste weight by combining RGB images with physics-informed metadata, including object dimensions, camera distance, and camera height. We also introduce Waste-Weight-10K, a real-world dataset containing 10,421 synchronized image-metadata collected from logistics and recycling sites. The dataset covers 11 waste categories and a wide weight range from 3.5 to 3,450 kg. Our model uses a Vision Transformer for visual features and a dedicated metadata encoder for geometric and category information, combining them with Stacked Mutual Attention Fusion that allows visual and physical cues guide each other. This helps the model manage perspective effects and link objects to material properties. To ensure stable performance across the wide weight range, we train the model using Mean Squared Logarithmic Error. On the test set, the proposed method achieves 88.06 kg Mean Absolute Error (MAE), 6.39% Mean Absolute Percentage Error (MAPE), and an R2 coefficient of 0.9548. The model shows strong accuracy for light objects in the 0-100 kg range with 2.38 kg MAE and 3.1% MAPE, maintaining reliable performance for heavy waste in the 1000-2000 kg range with 11.1% MAPE. Finally, we incorporate a physically grounded explanation module using Shapley Additive Explanations (SHAP) and a large language model to provide clear, human-readable explanations for each prediction.

Figures & tables

Study	Input Modality	Architecture	Dataset(s)	Geometry Modeling	Material Modeling	Scale Ambiguity Handling	Weight Range
[ 17 ]	Single RGB	Two-tower CNN (density + volume)	Amazon (150k), Household	Explicit (thickness masks + geometry)	Implicit (appearance-based)	None (requires dimensions)	Approximate 0.03-2 kg
[ 18 ]	Multi-view RGB	3D reconstruction + cylinder model + regression	100 industrial steel parts	Explicit (full 3D reconstruction)	Fixed (iron density)	Multi-view only	Not Specified
[ 20 ]	RGB	YOLOv8 + Support Vector regression	food ingredients images	Calibrated 2D shape-based geometry	Implicit (learned via regression)	None	Not Specified
[ 21 ]	Single RGB	Image2Mass + material embeddings	Amazon, MINC	Explicit (thickness + dimensions)	Broad categorical (MINC)	None (requires dimensions)	0.6-6 kg
[ 12 ]	Single RGB	Xception + CLIP + cross-attention	Amazon, ABO-500	Explicit (thickness + geometry features)	Explicit (182 materials)	None (fails without dimensions)	Not Specified
Ours	Single RGB + Physical & Camera Geometry	ViT + Metadata Encoder + Mutual Attention Fusion	Waste-Weight-10K (C&I)	Explicit (physics-informed geometric features)	Explicit (category-aware density reasoning)	Explicit (distance-aware, perspective-corrected)	3.5-3,450 kg

TABLE I: Comparative analysis of image-based mass and weight estimation methods, highlighting modality, architecture, geometric and material reasoning, scale handling, optimization strategy, and applicable weight range.

Fig. 1: Representative samples from the Waste-Weight-10K dataset. The figure illustrates the diversity in object shapes, textures, and lighting conditions captured across different commercial & industrial environments.

Fig. 2: Distribution of Waste Categories. The dataset includes many types of metal scrap collected from C&I environments.

Feature	Mean	Std	Min	Median	Max
Weight (kg)	751.93	841.50	3.5	210.0	3450.0
$L_{x}$ (m)	96.26	54.26	12.0	92.0	180.0
$L_{y}$ (m)	51.27	26.25	12.0	47.0	891.0
$L_{z}$ (m)	45.32	17.10	6.0	48.0	85.0
$D_{x}$ (m)	86.55	35.56	24.0	85.0	187.0
$D_{y}$ (m)	55.64	13.43	12.0	52.0	126.0

TABLE II: Summary of physical and geometric measurements (

N=10,421

). Here, the geometric measurements for each object’s 3D size are

L_{x},L_{y},L_{z}

, and

D_{x}

and

D_{y}

are the horizontal camera distance and lens height, respectively.

Fig. 3: Weight Distribution of the Dataset. The chart on the left shows the raw weights, while the chart on the right shows the weights after a log transformation, which simplifies the training process for the model.

Fig. 4: Scatter plot of Volume ( $m^{3}$ ) versus Weight (kg). The distribution highlights the complexity of the problem: small dense objects can be heavier than large voluminous ones, necessitating a multimodal approach.

Category	Count	Share (%)	Min (kg)	Max (kg)	Min Vol ( $m^{3}$ )	Max Vol ( $m^{3}$ )
Automotive Scrap	3,514	33.7%	3.5	1,610	0.0039	1.7820
Ferrous Metal	3,050	29.3%	9	1,850	0.0010	0.4189
Cardboard	1,094	10.5%	9.3	355	0.0084	0.2930
Rigid Plastic	799	7.7%	9	210	0.0284	0.0884
Wood	701	6.7%	52	3,450	0.0162	0.4885
General Trash	336	3.2%	14	28	0.0240	0.0790

TABLE III: Final Categorization of C&I Waste Categories with Weight Ranges.

Fig. 5: Overview of the proposed Multimodal Weight Predictor (MWP) framework. It integrates visual features with metadata through a Mutual Attention Fusion mechanism.

Aspect	Feature	Formula	Selection Rationale (Hybrid Strategy)
Size	log(Volume)	$\ln(1+L_{x}L_{y}L_{z})$	Primary Signal : High correlation ( $r=0.66$ ). Anchor feature.
	log(Surf. Area)	$\ln(1+2\sum L_{i}L_{j})$	Rejected : Collinear with Volume ( $r>0.99$ ). Redundant.
	log(Max Dim)	$\ln(1+\max(L_{i}))$	Shape Outlier : $r=0.59$ . Captures long beams/pipes distinct from cubes.
	log(Geo Mean)	$\ln(1+\sqrt[3]{L_{x}L_{y}L_{z}})$	Rejected : Mathematical duplicate of Volume ( $r=1.0$ ).
Material	Compactness	$L_{\min}/L_{\max}$	Density Proxy : $r=-0.29$ . Distinguishes dense bales from loose scrap.
Material	log(Vol/Surf)	$\ln(1+V/A_{s})$	Complexity : $r=0.64$ . Measures object ”hollowness”.

TABLE IV: Taxonomy of Engineered Physics-Informed Features. Selection utilizes a hybrid strategy: Correlation (

r

) for size magnitude and Domain Knowledge for material density and perspective correction.

Fig. 6: Architecture of our proposed extended visual transformer.

Fig. 7: Training and Validation Curves. The vertical line at Epoch 10 denotes the transition from the frozen to the fine-tuning phase. Note the sharp descent in MAE once visual features were unfrozen, validating the need for domain-specific texture adaptation.

Mass Category	Samples	MAE (kg)	MAPE (%)
Light (0–100 kg)	187	2.38	3.1
Medium (100–500 kg)	441	27.85	13.5
Heavy (1000–3500 kg)	824	164.93	11.1

TABLE V: Granular Performance Metrics by Weight Range.

Fig. 8: Error Analysis by Weight Range. The bar chart (Left Axis) shows the Mean Absolute Error (MAE) increases with mass, while the line chart (Right Axis) confirms the Mean Absolute Percentage Error (MAPE) remains stable across all categories.

Aspects	Variant	MAE	RMSE	MAPE
Loss	MSE Loss	62.45	75.10	19.42
	L1 Loss	51.30	95.20	14.15
	MSLE (Ours)	48.12	88.30	8.73
Fusion	No Fusion (Concat)	234.74	278.12	25.28
	One-Way (Vis $\to$ Meta)	76.50	132.40	13.40
	One-Way (Meta $\to$ Vis)	69.15	115.80	11.20

TABLE VI: Ablation of the impact of Loss Function, Fusion Strategy, Network Fusion Depth, and Visual Granularity. The default configuration (MSLE, Mutual Fusion, 2 Layers, ViT-B/16) acts as the baseline for comparison. MAE and RMSE are presented in kilograms, and MAPE in percentage.

Based	Model	MSLE	MAE	MAPE
CNN	VGG16	0.0018	112.35	14.82
	ResNet-50	0.0012	85.20	11.55
	DenseNet-121	0.0011	82.45	11.20
	InceptionV3	0.0010	79.80	10.85
	MobileNetV2	0.0013	88.60	12.05
	EfficientNet-B0	0.0009	75.40	10.25

TABLE VII: Comparison with state-of-the-art models. We compare our Multimodal Weight Predictor (MWP) against key CNN and Transformer baselines adapted for this regression task. Our approach achieves the lowest Loss and Error rates. MAE and MAPE are presented in kg and percentage, respectively.

Ref.	Dataset	Reported Performance
[ 17 ]	Amazon.com	mALDE: $0.47\pm 0.028$ , mAPE: $0.651\pm 0.071$
[ 18 ]	Industrial	Success rate: $92\%$
[ 20 ]	Food Images	[email protected]: $98.11$ , R2: $0.928$ , MAE: $4.76g$
[ 21 ]	Amazon.com	MSE: $1.943\pm 0.179$ , mALDE: $0.479\pm 0.011$
[ 12 ]	Amazon.com	mALDE: $0.457\pm 0.007$ , mAPE: $0.55\pm 0.014$
Ours	Waste-Weight-10K	MAE: 88.06 kg, RMSE: 181.52 kg, MAPE: 6.39%

TABLE VIII: Quantitative comparison of image-based mass and weight estimation methods. Metrics are reported as provided in the original studies and are not directly comparable due to dataset and task differences.

Multimodal Fusion

Multimodal fusion combines information from different data types to improve prediction, perception, and decision-making.

See all 771 papers

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models
Aug 24, 2026Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz +4Vision-Language Grounding Object Detection
United International University
PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models
Mar 17, 2026Hisayuki Yokomizo, Taiki Miyanishi, Yan Gang +3VLM Reasoning VLMs for Robotics
The University of Tokyo · Institute of Science Tokyo · National Institute of Informatics
MDWD: A Street-Level Dataset for Municipal Solid Waste Detection in Dense Urban Environments
Jul 31, 2026Andrea Filiberto Lucas, Mark Bugeja, Carl James Debono +1Object Detection
University of Malta

Aug 24, 2026J/K move · Enter open · S save

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models

Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz, Mahedi Hasan+3

United International University

Vision-Language Grounding Object Detection VLM Evaluation Object Hallucination in VLMs

Open paper AskPDF arXiv

Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquatic-waste datasets provide limited geographic coverage, sparse multi-instance annotations, and little supervision beyond boxes and labels. Compact vision-language models (VLMs) therefore remain insufficiently evaluated for jointly localizing, classifying, counting, and explaining floating waste. We introduce WADE, a reasoning-annotated benchmark containing 2,167 images from rural Bangladesh, 13,608 bounding boxes, and ten waste categories. Each annotation is associated with class-level recognition rules covering visual cues, likely confusions, and discriminative features. We evaluate six VLMs under zero-shot, two-shot, reasoning-guided, and fine-tuned settings using detection, counting, and hallucination metrics. For resource-efficient adaptation, we jointly fine-tune Qwen3-VL-2B on boxes, labels, and reasoning chains using QLoRA. Fine-tuning increases recall from 0.0248 to 0.2339 and F1 from 0.0257 to 0.2163, while reducing image-level hallucination from 0.6836 to 0.0883. However, over three-quarters of instances remain undetected, establishing WADE as a challenging benchmark for dense floating-waste grounding with compact VLMs.

Abstract

Multimodal Fusion

Explore similar work

WADE: A Reasoning-Annotated Benchmark for Multi-Instance Floating-Waste Grounding with Compact Vision-Language Models

PhysQuantAgent: An Inference Pipeline of Mass Estimation for Vision-Language Models

MDWD: A Street-Level Dataset for Municipal Solid Waste Detection in Dense Urban Environments