Organizations: Department of Computer Science and Engineering, United International University, Dhaka 1212, Bangladesh and also with Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Department of Data Science and Artificial Intelligence, Monash University, 3800, Melbourne, Australia, and also with Applied Artificial INtelligence and Intelligent Systems (AAIINS) Laboratory, Dhaka 1217, Bangladesh · Faculty of Science and Technology, Charles Darwin University, Darwin, NT 0810, Australia · Faculty of Arts and Society, Charles Darwin University, Casuarina, Darwin, NT 0810, Australia
Accurate weight estimation of commercial and industrial waste is important for efficient operations, yet image-based estimation remains difficult because similar-looking objects may have different densities, and the visible size changes with camera distance. Addressing this problem, we propose Multimodal Weight Predictor (MWP) framework that estimates waste weight by combining RGB images with physics-informed metadata, including object dimensions, camera distance, and camera height. We also introduce Waste-Weight-10K, a real-world dataset containing 10,421 synchronized image-metadata collected from logistics and recycling sites. The dataset covers 11 waste categories and a wide weight range from 3.5 to 3,450 kg. Our model uses a Vision Transformer for visual features and a dedicated metadata encoder for geometric and category information, combining them with Stacked Mutual Attention Fusion that allows visual and physical cues guide each other. This helps the model manage perspective effects and link objects to material properties. To ensure stable performance across the wide weight range, we train the model using Mean Squared Logarithmic Error. On the test set, the proposed method achieves 88.06 kg Mean Absolute Error (MAE), 6.39% Mean Absolute Percentage Error (MAPE), and an R2 coefficient of 0.9548. The model shows strong accuracy for light objects in the 0-100 kg range with 2.38 kg MAE and 3.1% MAPE, maintaining reliable performance for heavy waste in the 1000-2000 kg range with 11.1% MAPE. Finally, we incorporate a physically grounded explanation module using Shapley Additive Explanations (SHAP) and a large language model to provide clear, human-readable explanations for each prediction.
Figures & tables
Study
Input Modality
Architecture
Dataset(s)
Geometry Modeling
Material Modeling
Scale Ambiguity Handling
Weight Range
[ 17 ]
Single RGB
Two-tower CNN (density + volume)
Amazon (150k), Household
Explicit (thickness masks + geometry)
Implicit (appearance-based)
None (requires dimensions)
Approximate 0.03-2 kg
[ 18 ]
Multi-view RGB
3D reconstruction + cylinder model + regression
100 industrial steel parts
Explicit (full 3D reconstruction)
Fixed (iron density)
Multi-view only
Not Specified
[ 20 ]
RGB
YOLOv8 + Support Vector regression
food ingredients images
Calibrated 2D shape-based geometry
Implicit (learned via regression)
None
Not Specified
[ 21 ]
Single RGB
Image2Mass + material embeddings
Amazon, MINC
Explicit (thickness + dimensions)
Broad categorical (MINC)
None (requires dimensions)
0.6-6 kg
[ 12 ]
Single RGB
Xception + CLIP + cross-attention
Amazon, ABO-500
Explicit (thickness + geometry features)
Explicit (182 materials)
None (fails without dimensions)
Not Specified
Ours
Single RGB + Physical & Camera Geometry
ViT + Metadata Encoder + Mutual Attention Fusion
Waste-Weight-10K (C&I)
Explicit (physics-informed geometric features)
Explicit (category-aware density reasoning)
Explicit (distance-aware, perspective-corrected)
3.5-3,450 kg
TABLE I: Comparative analysis of image-based mass and weight estimation methods, highlighting modality, architecture, geometric and material reasoning, scale handling, optimization strategy, and applicable weight range.
Fig. 1: Representative samples from the Waste-Weight-10K dataset. The figure illustrates the diversity in object shapes, textures, and lighting conditions captured across different commercial & industrial environments.
Fig. 2: Distribution of Waste Categories. The dataset includes many types of metal scrap collected from C&I environments.
Feature
Mean
Std
Min
Median
Max
Weight (kg)
751.93
841.50
3.5
210.0
3450.0
Lx (m)
96.26
54.26
12.0
92.0
180.0
Ly (m)
51.27
26.25
12.0
47.0
891.0
Lz (m)
45.32
17.10
6.0
48.0
85.0
Dx (m)
86.55
35.56
24.0
85.0
187.0
Dy (m)
55.64
13.43
12.0
52.0
126.0
TABLE II: Summary of physical and geometric measurements ( N=10,421 ). Here, the geometric measurements for each object’s 3D size are Lx,Ly,Lz , and Dx and Dy are the horizontal camera distance and lens height, respectively.
Fig. 3: Weight Distribution of the Dataset. The chart on the left shows the raw weights, while the chart on the right shows the weights after a log transformation, which simplifies the training process for the model.
Fig. 4: Scatter plot of Volume ( m3 ) versus Weight (kg). The distribution highlights the complexity of the problem: small dense objects can be heavier than large voluminous ones, necessitating a multimodal approach.
Category
Count
Share (%)
Min (kg)
Max (kg)
Min Vol ( m3 )
Max Vol ( m3 )
Automotive Scrap
3,514
33.7%
3.5
1,610
0.0039
1.7820
Ferrous Metal
3,050
29.3%
9
1,850
0.0010
0.4189
Cardboard
1,094
10.5%
9.3
355
0.0084
0.2930
Rigid Plastic
799
7.7%
9
210
0.0284
0.0884
Wood
701
6.7%
52
3,450
0.0162
0.4885
General Trash
336
3.2%
14
28
0.0240
0.0790
TABLE III: Final Categorization of C&I Waste Categories with Weight Ranges.
Fig. 5: Overview of the proposed Multimodal Weight Predictor (MWP) framework. It integrates visual features with metadata through a Mutual Attention Fusion mechanism.
Aspect
Feature
Formula
Selection Rationale (Hybrid Strategy)
Size
log(Volume)
ln(1+LxLyLz)
Primary Signal : High correlation ( r=0.66 ). Anchor feature.
log(Surf. Area)
ln(1+2∑LiLj)
Rejected : Collinear with Volume ( r>0.99 ). Redundant.
log(Max Dim)
ln(1+max(Li))
Shape Outlier : r=0.59 . Captures long beams/pipes distinct from cubes.
log(Geo Mean)
ln(1+3LxLyLz)
Rejected : Mathematical duplicate of Volume ( r=1.0 ).
Material
Compactness
Lmin/Lmax
Density Proxy : r=−0.29 . Distinguishes dense bales from loose scrap.
TABLE IV: Taxonomy of Engineered Physics-Informed Features. Selection utilizes a hybrid strategy: Correlation ( r ) for size magnitude and Domain Knowledge for material density and perspective correction.
Fig. 6: Architecture of our proposed extended visual transformer.
Fig. 7: Training and Validation Curves. The vertical line at Epoch 10 denotes the transition from the frozen to the fine-tuning phase. Note the sharp descent in MAE once visual features were unfrozen, validating the need for domain-specific texture adaptation.
Mass Category
Samples
MAE (kg)
MAPE (%)
Light (0–100 kg)
187
2.38
3.1
Medium (100–500 kg)
441
27.85
13.5
Heavy (1000–3500 kg)
824
164.93
11.1
TABLE V: Granular Performance Metrics by Weight Range.
Fig. 8: Error Analysis by Weight Range. The bar chart (Left Axis) shows the Mean Absolute Error (MAE) increases with mass, while the line chart (Right Axis) confirms the Mean Absolute Percentage Error (MAPE) remains stable across all categories.
Aspects
Variant
MAE
RMSE
MAPE
Loss
MSE Loss
62.45
75.10
19.42
L1 Loss
51.30
95.20
14.15
MSLE (Ours)
48.12
88.30
8.73
Fusion
No Fusion (Concat)
234.74
278.12
25.28
One-Way (Vis → Meta)
76.50
132.40
13.40
One-Way (Meta → Vis)
69.15
115.80
11.20
TABLE VI: Ablation of the impact of Loss Function, Fusion Strategy, Network Fusion Depth, and Visual Granularity. The default configuration (MSLE, Mutual Fusion, 2 Layers, ViT-B/16) acts as the baseline for comparison. MAE and RMSE are presented in kilograms, and MAPE in percentage.
Based
Model
MSLE
MAE
MAPE
CNN
VGG16
0.0018
112.35
14.82
ResNet-50
0.0012
85.20
11.55
DenseNet-121
0.0011
82.45
11.20
InceptionV3
0.0010
79.80
10.85
MobileNetV2
0.0013
88.60
12.05
EfficientNet-B0
0.0009
75.40
10.25
TABLE VII: Comparison with state-of-the-art models. We compare our Multimodal Weight Predictor (MWP) against key CNN and Transformer baselines adapted for this regression task. Our approach achieves the lowest Loss and Error rates. MAE and MAPE are presented in kg and percentage, respectively.
TABLE VIII: Quantitative comparison of image-based mass and weight estimation methods. Metrics are reported as provided in the original studies and are not directly comparable due to dataset and task differences.
Floating waste in inland waterways threatens aquatic ecosystems and requires timely monitoring under cluttered, multi-object conditions. Existing aquatic-waste datasets provide limited geographic coverage, sparse multi-instance annotations, and little supervision beyond boxes and labels. Compact vision-language models (VLMs) therefore remain insufficiently evaluated for jointly localizing, classifying, counting, and explaining floating waste. We introduce WADE, a reasoning-annotated benchmark containing 2,167 images from rural Bangladesh, 13,608 bounding boxes, and ten waste categories. Each annotation is associated with class-level recognition rules covering visual cues, likely confusions, and discriminative features. We evaluate six VLMs under zero-shot, two-shot, reasoning-guided, and fine-tuned settings using detection, counting, and hallucination metrics. For resource-efficient adaptation, we jointly fine-tune Qwen3-VL-2B on boxes, labels, and reasoning chains using QLoRA. Fine-tuning increases recall from 0.0248 to 0.2339 and F1 from 0.0257 to 0.2163, while reducing image-level hallucination from 0.6836 to 0.0883. However, over three-quarters of instances remain undetected, establishing WADE as a challenging benchmark for dense floating-waste grounding with compact VLMs.
Md. Asaduzzaman Shuvo, Ahsan Farabi, Md. Abdul Ahad Minhaz +4
Vision-Language Models (VLMs) are increasingly applied to robotic perception and manipulation, yet their ability to infer physical properties required for manipulation remains limited. In particular, estimating the mass of real-world objects is essential for determining appropriate grasp force and ensuring safe interaction. However, current VLMs lack reliable mass reasoning capabilities, and most existing benchmarks do not explicitly evaluate physical quantity estimation under realistic sensing conditions. In this work, we propose PhysQuantAgent, a framework for real-world object mass estimation using VLMs, together with VisPhysQuant, a new benchmark dataset for evaluation. VisPhysQuant consists of RGB-D videos of real objects captured from multiple viewpoints, annotated with precise mass measurements. To improve estimation accuracy, we introduce three visual prompting methods that enhance the input image with object detection, scale estimation, and cross-sectional image generation to help the model comprehend the size and internal structure of the target object. Experiments show that visual prompting significantly improves mass estimation accuracy on real-world data, suggesting the efficacy of integrating spatial reasoning with VLM knowledge for physical inference.
Hisayuki Yokomizo, Taiki Miyanishi, Yan Gang +3
The University of Tokyo · Institute of Science Tokyo · National Institute of Informatics
Automated visual monitoring of urban environments is a growing Computer Vision research area, but municipal solid waste detection remains under-represented in dedicated benchmark resources. Existing waste-related datasets predominantly address individual litter detection, aerial imagery, or image-level classification, and none simultaneously provide street-level imagery, instance-level localization, and categorization of domestic waste streams within a structured municipal collection context. This paper introduces the Maltese Domestic Waste Dataset (MDWD), a street-level benchmark comprising 3,697 high-resolution images and 11,461 manually annotated instances across five domestic waste categories representative of Malta's municipal collection system. The dataset captures substantial variation in location, illumination, object scale, occlusion, and urban context. To establish reproducible baselines, a cross-architecture benchmark is conducted across multiple generations of the YOLO family and a transformer-based detector. On the test set, RF-DETR-M achieves the strongest overall performance with an mAP50 of 94.49% and an F1-score of 93.56%, whilst smaller-capacity variants maintain competitive accuracy at substantially reduced parameter counts. These results indicate that MDWD supports effective training across both compact real-time detectors and transformer-based models, establishing a benchmark for future research in vision-based municipal waste monitoring.
Andrea Filiberto Lucas, Mark Bugeja, Carl James Debono +1