AgenticTactileVLA: Contact-Guided Execution-Time Supervision for Generalizable Dexterous Manipulation without VLA Retraining
Authors: Elizaveta Semenyakina, Ivan Snegirev, Mikhail Kiselev, Miguel Altamirano Cabrera, Artem Lykov, Hajira Amjad, Dzmitry Tsetserukou
Organizations: Intelligent Space Robotics Lab, Skolkovo Institute of Science and Technology, Moscow, Russian Federation. · R&D Center, MWS, Moscow, Russian Federation.
Vision-language-action policies may predict a transferable manipulation strategy yet fail to realize it reliably on the encountered object: objects compatible with the same grasp differ in geometry and compliance, and visual feedback degrades under closure occlusion. AgenticTactileVLA is presented as an execution-time supervisor that shifts part of object-specific adaptation from prediction to physical interaction. A fixed VLA provides the approach and hand targets; the supervisor decides whether to remain transparent, refine finger flexion, retain or release the corrected configuration, return control to the VLA for retry, or select a compliant hand-control regime. It uses finger-position and motor-effort feedback as proprioceptive contact evidence and requires neither tactile sensors nor VLA retraining. On a Unitree G1 with a BrainCo Revo2 hand, a randomized matched-block evaluation on five objects held out from VLA training yields 61.3% completion for the base VLA, 72.0% for unconditional close-to-stall control, and 84.0% for the supervisor under a shared budget; the gain is positive on every object and persists under moderate pose perturbations. Ablations show the gain is not explained by extended closure alone, and that selective triggering reduces correction episodes by 65.7% with no detected change in completion. A retention audit shows acceptance predicts retention in 88.9% of held-out cases, while compliant objects expose conservative false rejection. A thin-walled-cup study demonstrates contextual routing to compliant control, matching an always-compliant reference. These results suggest that contact-guided execution-time adaptation can improve the object-level generalization of a fixed VLA to held-out objects by adapting physical realization without object-specific retraining.
Figures & tables
Fig. 1: From VLA prediction to execution-time physical adaptation. The fixed VLA provides the approach and initial hand configuration, while the execution agent adapts finger closure to the encountered object.
Fig. 2: System architecture of AgenticTactileVLA. The fixed VLA provides the nominal manipulation strategy, while the execution supervisor monitors its physical realization and intervenes only when adaptation is required.
State
bt
Override
Transition / exit condition
Idle
0
no
upcoming closure in the action chunk
Armed
0
no
terminal grasp assessment
Adjust
1
yes
grasp verdict or timeout
Hold
0
yes
policy reopens the hand
TABLE I: Execution-agent states. bt gates body-reference progression; the override column indicates whether the agent replaces the nominal hand target.
Object
VLA
VLA + stall
Supervisor
Cube
9/15
11/15
13/15
Fish
9/15
11/15
13/15
Soy sauce bottle
11/15
12/15
14/15
Thermos lid
7/15
8/15
10/15
Hand cream tube
10/15
12/15
13/15
Pooled
46/75
54/75
63/75
TABLE II: Fixed-budget task completion under randomized, interleaved conditions. All conditions use 15 trials per held-out object; intervals are 95% Wilson.