Tactile sensing is particularly valuable for contact-rich robotic manipulation. Recent work has made substantial progress in tactile representation learning for robotic manipulation. However, similar tactile observations can arise from interaction conditions with very different task-risk implications, such as sensor noise, task-necessary variations, and emerging undesirable contact. Without context-grounded risk information, these cases can be ambiguous to downstream policies, leading to unnecessary corrections to benign variations or delayed responses to genuinely risky contact. To address this limitation, we propose Risk-Aware Tactile Encoding (RATE), which learns tactile representations that encode task-conditioned interaction risk. Specifically, history-conditioned prediction captures interaction context, while alert supervision associates this context with task-conditioned risk. The learned risk-aware representation complements conventional tactile features through a lightweight residual adapter. Experiments in both simulation and the real world demonstrate substantial improvements in task success, with controlled ablations confirming the complementary benefits of alert-guided learning and predictive temporal modeling.
Figures & tables
Figure 1: RATE in contact-rich manipulation: corrective behavior and task performance. (a) Representative interaction sequence: when the interaction reaches critical contact, baseline methods may continue toward failure, whereas RATE enables a corrective response toward successful execution. (b) Success rates across all eight simulation tasks, comparing RATE with representative tactile and large-scale policy baselines. RATE achieves the best or jointly best performance on six of the eight tasks. (c) Success rates on three real-robot tasks, including USB insertion, bottle-cap screwing, and plug insertion, where RATE achieves the highest success rate on all three tasks.
Figure 2: Overview of RATE representation learning and policy adaptation. RATE learns a complementary risk-aware representation from tactile history, jointly shaped by short-horizon future contact-change prediction and future alert supervision. During policy adaptation, the learned representation is mapped to a bounded residual and fused with conventional tactile features through a lightweight residual pathway. The resulting tactile representation, together with camera observations and robot state, conditions the downstream manipulation policy. The tactile encoders and policy backbone remain frozen during adaptation, and all auxiliary prediction heads are used only for training.
Method
Params.
Contact-Rich Interaction
Pose Reasoning
Shape Perception
Avg.
Pull-out Key
Insert Hole
Insert HDMI
Insert Tube
Lift Bottle
Lift Can
Put Bottle
Grasp Classify
ACT (Vision Only) [ 22 ]
0.08B
28.0
19.0
15.0
45.0
42.0
20.0
28.0
50.0
30.9
ACT+UniVTAC [ 4 ]
0.10B
46.0
24.0
28.0
56.0
71.0
29.0
31.0
99.0
48.0
VITaL [ 2 ]
0.10B
47.0
25.0
6.0
34.0
72.0
8.0
32.0
100.0
40.5
π0.5 [ 23 ]
3.3B
35.0
25.0
8.0
74.0
100.0
6.0
34.0
49.0
41.4
StarVLA- α [ 24 ]
4B
51.0
52.0
24.0
69.0
32.0
65.0
88.0
68.0
56.1
Table I: Simulation success rates (%) on the eight-task UniVTAC benchmark. Best results are bold. Published ACT, ACT+UniVTAC, and VITaL success rates are reported in UniVTAC [ 4 ] . The success rates of π0.5 , StarVLA- α , and GigaWorld-Policy are reported in N0 -VTLA [ 13 ] . Published results are included for reference and may differ in demonstration count and observation configuration. ACT+UniVTAC (Retrained) is retrained and evaluated by us under the same configuration as RATE.
Alert Supervision
Context
Contact-Rich Interaction
Pose Reasoning
Shape Perception
Avg.
Discrete
Continuous
Transition- Aware
Pull-out Key
Insert Hole
Insert HDMI
Insert Tube
Lift Bottle
Lift Can
Put Bottle
Grasp Classify
–
–
–
–
51.0
30.0
16.0
49.0
51.0
49.0
7.0
99.0
44.0
✓
–
–
–
65.0
31.0
31.0
59.0
67.0
62.0
34.0
100.0
56.1
–
✓
–
–
81.0
32.0
18.0
66.0
61.0
67.0
35.0
100.0
57.5
–
–
✓
–
78.0
41.0
14.0
66.0
68.0
52.0
29.0
100.0
56.0
–
–
–
✓
68.0
46.0
25.0
56.0
73.0
68.0
12.0
100.0
56.0
Table II: Ablation results for RATE on the eight-task UniVTAC benchmark. All variants share the same task-specific tactile base. The table evaluates alert supervision using Discrete Ordinal, Continuous Severity, and Transition-Aware Continuous formulations, while context denotes interaction context learned through history-conditioned prediction. The final row corresponds to Full RATE, combining continuous severity alert supervision with context. Avg. denotes the eight-task macro average. Best results are bold.
Method
USB Insertion
Bottle Cap Screwing
Plug Insertion
SR
CSSR
SR
CSSR
SR
CSSR
ACT (Vision Only)
0.0
0.00
15.0
33.33
30.0
33.33
ACT+UniVTAC
30.0
0.00
35.0
85.71
50.0
70.00
π0.5
40.0
62.50
45.0
55.56
35.0
57.14
RATE (Ours)
75.0
93.33
85.0
100.00
60.0
83.33
Table III: Real-world evaluation over 20 trials per task. Task Success Rate (SR) and Contact-Safe Success Rate (CSSR) are reported in separate columns, in percent. Best results are bold.
Figure 3: Real-world contact-rich manipulation tasks. Rows from top to bottom show USB Insertion, Bottle Cap Screwing, and Plug Insertion. Columns show successive stages of each task from left to right.
Shanghai Key Lab of Intelligent Information Processing, College of Computer Science and Artificial Intelligence, Fudan University · College of Intelligent Robotics and Advanced Manufacturing, Fudan University · TeleAl, China Telecom +2