Fine-grained recognition of visually similar industrial parts is challenging when classes differ primarily in physical dimensions. Normalizing detected object crops to a fixed input size suppresses absolute scale, while CAD models and large class-specific datasets may be unavailable in evolving industrial inventories. We present CALIPER, a model-free RGB-D framework that couples support-based appearance matching with metric size evidence. Each training class is onboarded from a single turntable RGB-D video and one to two labeled real images; 3D reconstruction provides novel-view appearance support, while aligned depth yields a class-specific metric size profile. At inference, a coarse YOLOv8n-seg model localizes parts, and a frozen DINOv2 backbone with an episodically trained embedding head performs fine-grained support matching. Margin-conditioned metric fusion activates probabilistic size evidence only for appearance-ambiguous decisions. New classes are enrolled from a small RGB-D support set without updating network parameters. We evaluate CALIPER on 18 visually similar industrial parts: 16 classes are used for training, while two screws are reserved for training-free enrollment. CALIPER achieves 88.2% closed-set accuracy with 99.8% localization recall and 85.7% overall accuracy after 10-shot enrollment of the two unseen screws. Metric fusion improves unseen-class accuracy by up to 37.4 percentage points without statistically significant degradation of the original inventory. Robot-arm deployment identifies 17/18 parts without deployment-specific retraining.
Figures & tables
Figure 1: Overview of CALIPER. (A) Training classes are onboarded to build appearance support banks, a coarse detector, and class-specific metric size profiles; limited real data provide sim-to-real supervision and calibrate the fusion parameters. (B) An unseen class is enrolled from SI10 RGB-D images by forming a new support bank and live size profile, without retraining. (C) At inference, the coarse detector provides object crops and masks for appearance matching and metric measurement, respectively. Existing and newly enrolled banks/profiles are combined, and margin-conditioned fusion produces the final fine-grained identity. Here, E and N denote existing and newly enrolled class sets; TV1 , RLI12 , BV2 , SI10 are different data tiers explained in Sec. 3 .
Figure 2: The benchmark includes 18 classes across three coarse categories – two gears, seven nuts, and nine screws. screw_h (M14) and screw_i (M20) are the two unseen enrollment probes; they appear nearly identical in the visual feature space but carry distinct sizes.
Figure 3: Directional separability among fine-grained classes within three coarse categories – gears, nuts, and screws. Each cell (A,B) reports S(A→B) , measuring how well class A is separated from class B in the head-embedding space using onboarding ( 16 training classes) and enrollment ( 2 unseen screws) data. Lower values indicate greater visual similarity and higher confusion potential; diagonal cells are empty because self-separability is undefined.
Method
K=1
K=5
K=10
One-stage YOLO *
63.4
63.4
63.4
Raw DINOv2
39.7
71.9
75.8
Linear classifier *
85.3
85.3
85.3
ProtoNet
87.3
88.3
87.6
CALIPER, rendered support
87.9
87.9
88.2
CALIPER, real support
88.2
87.4
88.2
Table 1: Closed-set accuracy (%) on the 16 training classes. Support-based results are averaged over 100 random draws. The default CALIPER configuration is marked in bold font.
Overall
Training16
Unseen2
Method
K=1
K=5
K=10
K=1
K=5
K=10
K=1
K=5
K=10
One-stage YOLO
61.4
53.8
55.1
63.6
53.4
53.4
42.9
56.9
68.4
Raw DINOv2
37.2
64.1
67.5
36.3
66.2
69.7
44.2
47.4
49.4
Linear classifier
71.5
78.6
78.8
77.6
83.0
81.1
20.3
41.7
60.0
ProtoNet
79.1
80.8
81.1
85.3
87.0
86.3
27.7
29.8
38.0
CALIPER
83.3
85.1
85.7
87.3
86.6
86.9
50.0
72.5
75.3
Table 2: Accuracy (%) after enrolling two unseen classes. Support-based results are averaged over 100 random support draws. For methods sharing CALIPER’s coarse detector, Overall , Training16 , and Unseen2 contain 557 , 497 , and 60 matched detections, respectively. One-stage YOLO is evaluated on detections from its retrained fine-grained detector, trained once per K .
K=1
K=5
K=10
Comparison
b
c
p
b
c
p
b
c
p
CALIPER vs Linear classifier
Overall
94
24
<0.001
64
23
<0.001
62
22
<0.001
Training16
73
19
<0.001
40
16
0.002
45
14
<0.001
Unseen2
21
5
0.002
24
7
0.003
17
8
0.108
CALIPER vs ProtoNet
Table 3: McNemar’s exact test ( b , c , p ) for CALIPER against ProtoNet and the Linear classifier in the 18-class setting of Table 2 .
16-class Training16
18-class Overall
18-class Training16
18-class Unseen2
Method
K=1
K=5
K=10
K=1
K=5
K=10
K=1
K=5
K=10
K=1
K=5
K=10
No metric fusion
87.1
87.3
88.3
78.5
80.4
81.2
84.8
85.5
86.5
26.8
38.3
37.9
With metric fusion
88.1
87.8
88.2
83.3
85.1
85.7
87.3
86.6
86.9
50.0
72.5
75.3
Δ
+1.0
+0.5
−0.1
+4.8
+4.7
+4.5
+2.5
+1.1
+0.4
+23.2
+34.2
+37.4
Table 4: Ablation of margin-conditioned metric fusion. Accuracy (%) is reported with and without metric fusion for the 16-class closed-set and 18-class enrollment settings, averaged over 100 random support draws; Δ denotes the change due to fusion.
Figure 4: Effect of training-free enrollment on existing and unseen classes. Accuracies are averaged over 100 random support draws for each K . Training16 (blue) is evaluated before enrollment (16-way, dashed) and after adding the two probes (18-way, solid). Unseen2 (red) reports enrolled-class accuracy in the 18-way setting.
Figure 5: Robot-arm deployment. (a,b) A seven-part scene and classification. (c,d) Motion above nut_b and to the predefined drop pose; no gripper is used. (e,f) Reclassification after one object is removed. (g,h) Additional scenes; (h) shows the only error (orange bounding box), screw_h → screw_c . All panels use frozen K=10 with no deployment-specific retraining or recalibration.
True class
screw_h
screw_i
screw_c
Other
screw_h , no fusion
42.5
31.3
22.9
3.3
screw_i , no fusion
39.9
33.3
10.9
15.9
screw_h , fused
82.0
0.0
14.7
3.3
screw_i , fused
16.7
68.7
0.0
14.6
Table 5: Row-normalized confusion (%) for the two enrolled classes at K=10 in the 18-class setting, with and without fusion.
Fine industrial robotic manipulation requires 6D pose, scale, and an operation-ready 3D model. Existing industrial pose estimation relies on CAD models, object-specific training data, or stable workcells, limiting use with unseen objects; zero-shot methods suffer from single-view occlusion, generative bias, and scale uncertainty. We present OV-PEAR, an observation-verified framework for 3D reconstruction and pose-scale estimation of unknown industrial objects. Its three-stage object-centric pipeline: (1) segments RGB-D instances and generates object-level 3D hypotheses; (2) anchors them to the robot frame via appearance-aware view retrieval, explicit scale search, and visibility-sensitive registration; (3) back-projects multi-view RGB-D evidence into the object-canonical frame to verify voxel occupancy, suppress hallucinations, and produce reusable trusted object models. On BOP-Industrial, OV-PEAR achieves 0.657 average AR and improves pose and voxel-level geometric consistency in real scenes; real-robot connector assembly reaches 99.67% grasp and 84.00% assembly success. The model library and pose-scale log enable short-term object-level state maintenance at 0.02 ms/frame.
Guoqin Tang, Qingxuan Jia, Yichen Tan +4
All authors are with the School of Intelligent Engineering and Automation, Beijing University of Posts and Telecommunications, Beijing 100876, China.
How far a pushed object slides depends on its mass and friction, which no single image reveals. Pretrained visual encoders are increasingly used as the perception front end of world models for manipulation, and their physical competence is assessed with perturbation benchmarks and linear probes, almost always in a clean, fixed-camera scene. We show that these assessments cannot distinguish an encoder that infers physics from one that does not. CALIPER (calibrate, then predict) is a direct test: an object of unknown mass and friction is struck twice at known speeds, a third strike is shown only up to the moment of contact, and a linear readout on frozen features must predict how far the object slides. Swapping in another object's calibration clips checks that the evidence is actually used. Across 2,000 simulated episodes and eight representations, from V-JEPA 2 to a randomly initialised ViT and raw pixels, calibration adds +0.50 R^2 and the swap removes it. Yet in the clean scene every representation lands within 0.02 R^2 of the ceiling set by true simulator state, because a fixed camera exposes the object's displacement directly in pixel coordinates. Resampling camera, lighting, and clutter for every clip spreads the same representations across 0.50 R^2; when the readout chooses a push speed for a goal distance, V-JEPA 2 misses by 4 mm and the random ViT by 20 mm, no better than ignoring the object. Linear probes track none of this: a change in frame aggregation moves a probe more than pretraining does, and erasing the probed mass direction from the same representation costs nothing in one scene and 0.35 R^2 in the other. Whether a benchmark can rank models is an empirical property, and we give three checks that establish it.
3D MLLMs recognize objects via pattern matching, which fails when geometrically similar objects require functional reasoning to disambiguate (e.g., dresser vs. wardrobe). Our per-sample Geometric Confusion Score (GCS) exposes this failure: baseline accuracy is 33.7% on ModelNet40's most ambiguous 20% versus 84.0% on its least ambiguous 20%. We identify a function-as-annotation pattern, in which functional language decorates a known identity instead of supporting inference, and hypothesize that reversing this information flow should concentrate improvements on the most ambiguous samples. PFIR (Part-level Functional Inference for Recognition) reverses that flow. Built on PartNeXt part segmentation, its data pipeline produces 507K QA pairs with functional CoTs (geometry → function → identity), and its progressive curriculum culminates in dual-objective GRPO: strict functional reasoning and descriptive grounding for Objaverse's diverse distribution. The two RL objectives yield nearly orthogonal task vectors (cos=0.056), enabling composition via Task Arithmetic. Under a strict dual OOD protocol (Objaverse → ModelNet40; XYZRGB → XYZ-only), PFIR raises zero-shot accuracy from 60.5% to 66.7%, with gains concentrated 2.6× more on geometrically ambiguous samples---supporting part-level functional reasoning as an effective route to more generalizable 3D recognition. Code: https://github.com/this233/pfir-3d.
Weihong Li, Hao Jiang, Yaohua Zha +2
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · College of Computer Science and Software Engineering, Shenzhen University, Shenzhen, China