Edge-Native Temporal--Graph Dual-Expert Inference for Dexterous Grasp Classification
Authors: Jiajian Zheng, Bin Zhang, Wen Fan, Yanan Liu
Organizations: School of Microelectronics Shanghai University Shanghai, 201800, China · Department of Bioengineering Imperial College London London SW7 2AZ, United Kingdom
Multi-finger hands contact objects at different times, so a global close command can overload early-contact fingers while others are still moving. Dense tactile arrays address this problem but add hardware, calibration, bandwidth, and computation. We present an edge-native pipeline that uses only built-in joint-angle and pre-use-calibrated actuator-load feedback to recognize the grasped object and predict an independent Stop event for each finger. Four samples and their first differences form a 4-by-20 contact-evolution window. At its core is a heterogeneous dual-expert architecture, termed DualExpert, that assigns complementary tasks to a Temporal Convolutional Network (TCN) and a Finger Graph Filter Network (FGFN). The TCN models short-term contact evolution and supplies all five Stop probabilities, while the FGFN encodes finger topology and contributes only to object classification. Fixed 0.7/0.3 class-probability fusion reaches 93.78% trajectory-level classification accuracy on 225 held-out trials, improving the standalone TCN and FGFN by 4.00 and 2.22 percentage points, respectively. The TCN Stop path attains 95.25% F1. The dataset contains 1,500 automatically collected trajectories from five object instances. In Flash deployment, mixed accelerator execution retains 93.78% classification accuracy and 94.84% Stop F1 with 2.890ms mean inference time and 9.172ms mean sample-to-stop latency. Across 50 physical trials, the system achieves 96.00% recognition and 98.00% grasp success while the complete embedded board draws approximately 0.80~W.
Figures & tables
Fig. 1: Architectures of the complementary experts: (a) TCN96b5 models four-frame contact evolution and predicts object class and five-finger Stop probabilities; (b) FGFN80 injects the fixed finger topology through low/high graph responses for more balanced object classification.
Fig. 2: Heterogeneous DualExpert. The topology-aware FGFN complements temporal classification, while control-sensitive Stop outputs remain on TCN96b5.
Fig. 3: On-board per-finger grasp flow. Measured feedback forms each new contact frame; model-triggered Stop states are monotonic while other fingers continue incremental closure.
Fig. 4: Experimental platform and evaluated instances. From left to right: RH56 hand, plastic bottle, paper box, sponge, paper cup, and paper roll.
Fig. 5: Board-side streaming architecture used in the deployment experiments. Samples enter temporary SRAM once; feature construction and inference execute locally with immutable model parameters stored in Flash.
Model
Class acc. (%)
Macro F1 (%)
Stop F1 (%)
Params
Mean (ms)
P95 (ms)
MLP
68.00
67.25
78.02
30,105
0.261
0.481
1D CNN
77.78
77.31
82.64
29,258
0.648
1.126
GRU
80.00
79.59
82.64
46,282
0.301
0.445
LSTM
84.44
84.32
82.46
60,106
0.405
0.758
TCN96b5
89.78
89.62
95.25
65,194
1.490
2.580
FGFN80
91.56
91.55
81.92
57,375
1.223
1.792
TABLE I: Held-out test performance and computational cost
Fig. 6: Classification complementarity of TCN96b5, FGFN80, and DualExpert. The upper panel reports classification accuracy; the lower heatmap reports per-instance class F1.
Fig. 7: Per-finger Stop F1 for the TCN96b5 path used unchanged by DualExpert and the independently evaluated FGFN80 Stop head.
Execution
Class acc. (%)
Stop F1 (%)
Mean (ms)
PyTorch
93.78
95.25
2.314
ONNX FP32
93.78
95.16
0.207
Cortex-M55 only
93.33
94.17
3.949
Neural-ART mixed
93.78
94.84
2.890
TABLE II: Computer and target execution summary
Object
Size (mm)
Mass (g)
Trials
Correct recognition
Successful grasps
Plastic bottle
∅59×161
20.0
10
10
10
Paper box
76.8×52.5×136.4
18.5
10
10
10
Sponge
72.0×56.1×104.0
4.6
10
9
10
Paper cup
∅72.4/∅53.5×123.3
41.9
10
10
10
Paper roll
∅40.6×150.2
33.4
10
9
9
Overall
–
–
50
48
49
TABLE III: Physical objects and closed-loop outcomes