GPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation
Organizations: Tsinghua University · Amap, Alibaba Group
Abstract
Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.
Figures & tables
| Model | Lift Bottle | Pull-out Key | Lift Can | Put Bottle | Insert Hole | Insert HDMI | Insert Tube | Grasp Classify | Avg. |
| ACT | 42 | 28 | 20 | 28 | 19 | 15 | 45 | 50 | 30.9 |
| VITaL | 72 | 47 | 8 | 32 | 25 | 6 | 34 | 100 | 40.5 |
| 92 | 36 | 59 | 31 | 26 | 3 | 34 | 48 | 41.1 | |
| FTP-pi0.5 | 77 | 30 | 26 | 19 | 47 | 4 | 72 | 97 | 46.5 |
| UniVTAC-ACT | 71 | 46 | 29 | 31 | 24 | 28 | 56 | 99 | 48.0 |
| FTP-1 | 97 | 48 | 65 | 47 | 64 | 3 | 79 | 100 | 62.9 |
| Model | Weight Class. | Hardness Class. | Roughness Class. | Roughness Regrasp | Empty-Can Selection | Grasp Chip | Gear Assembly | Pull Drawer | Insert USB | Insert Block v1 | Insert Block v2 | Avg. |
| StarVLA (Qwen3) | 44 | 38 | 42 | 35 | 32 | 10 | 26 | 35 | 4 | 0 | 0 | 24.2 |
| StarVLA (Cosmos) | 51 | 67 | 94 | 9 | 13 | 1 | 33 | 22 | 3 | 1 | 3.3 | 27.0 |
| Lingbot-VA | 50 | 44 | 39 | 40 | 44 | 22 | 59 | 15 | 2 | 2 | 6 | 29.4 |
| FastWAM | 49 | 84 | 67 | 7 | 9 | 50 | 51 | 88 | 6 | 0 | 0 | 37.4 |
| XVLA | 28 | 30 | 56 | 47 | 45 | 42 | 84 | 76 | 2 | 3 | 0.7 | 37.6 |
| Xiaomi-Robotics | 52 | 46 | 54 | 62 | 38 | 24 | 73 | 84 | 5 | 2 | 0 | 40.0 |
| Input | Input tokens | Cached input | Output tokens | Total tokens | CNY |
| V | 127,250,755 | 104,837,109 | 2,283,555 | 129,534,310 | 3,102.25 |
| V+T | 203,911,092 | 170,693,217 | 2,521,218 | 206,432,310 | 4,402.80 |
| Total | 331,161,847 | 275,530,326 | 4,804,773 | 335,966,620 | 7,505.05 |
| Input | Requests | Mean | Median | P90 | P95 | P99 |
| V | 3,927 | 22.8 | 15.4 | 46.3 | 59.1 | 110.7 |
| V+T | 4,890 | 21.1 | 14.0 | 44.0 | 56.8 | 96.6 |
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | Shots | Segments | Total actions | Cameras | |
| UniVTAC | |||||
| Lift Bottle | 1 | 12 | 10 | 120 | Fixed |
| Pull-out Key | 1 | 12 | 5 | 60 | Fixed |
| Lift Can | 1 | 12 | 5 | 60 | Fixed + wrist |
| Put Bottle | 1 | 12 | 10 | 120 | Fixed |
| Insert Hole | 1 | 12 | 5 | 60 | Fixed |
Robotic Manipulation
Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.
See all 1,671 papers