cs.ROOct 19, 2026
SaveGPT-6 Astra Is a Native Visuo-Tactile Policy for Contact-Rich Manipulation
Organizations: Tsinghua University · Amap, Alibaba Group
Abstract
Recent advances in multimodal models have enabled GPT-6 Astra to perform robotic manipulation through in-context learning. Yet contact-rich control and tactile perception are often regarded as strengths of specialist policies, leaving open whether Astra can directly understand and use tactile observations. We provide, to our knowledge, the first demonstration that Astra directly interprets vision-based tactile images and uses their contact cues for manipulation. Tactile RGB images enter through its existing visual interface, enabling few-shot closed-loop control on nineteen simulated UniVTAC and OpenViTac tasks with 210 sampled scene seeds per setting. With visuo-tactile input, Astra achieves mean success rates of 93.8% and 81.8%, exceeding the strongest reported specialist baselines in our comparison by 22.9 and 13.1 percentage points, respectively. Vision-only performance also leads these baselines, reaching 76.3% and 80.6%. Trajectory analysis reveals active probing, cautious grasping, contact-guided alignment, and recovery, alongside difficulties with tangential loading and corrective execution. Our findings challenge the exclusivity of specialist policies in contact-rich manipulation and show that fine-grained vision can partially compensate for missing touch. They also identify vision-based tactile sensing as an interface for scaling contact understanding with foundation models and reveal signs of benchmark saturation. These results motivate joint visual-tactile pretraining, harder contact-rich evaluations, and physical-robot validation.
Figures & tables
Figure 1: Benchmark-level results. Task-averaged success rates (%). Astra uses vision alone (blue) or vision with tactile RGB images (red). Grey bars show published specialist baselines. Models are ordered by their reported mean within each benchmark. Task-level results and evaluation scope appear in Section 5 .
Figure 2: Direct tactile-image input for contact-guided manipulation. Tactile RGB images and scene observations enter Astra through its visual input interface. Demonstrations and online interaction memory connect contact evidence to actions and outcomes. Astra generates end-effector and gripper commands; benchmark execution returns observations and a receipt for the next decision. The displayed fixed-camera and tactile images are actual inputs from Insert Hole, seed 1000028. Vision-only evaluation omits the tactile images throughout the context.
| Model | Lift Bottle | Pull-out Key | Lift Can | Put Bottle | Insert Hole | Insert HDMI | Insert Tube | Grasp Classify | Avg. |
| ACT | 42 | 28 | 20 | 28 | 19 | 15 | 45 | 50 | 30.9 |
| VITaL | 72 | 47 | 8 | 32 | 25 | 6 | 34 | 100 | 40.5 |
| 92 | 36 | 59 | 31 | 26 | 3 | 34 | 48 | 41.1 | |
| FTP-pi0.5 | 77 | 30 | 26 | 19 | 47 | 4 | 72 | 97 | 46.5 |
| UniVTAC-ACT | 71 | 46 | 29 | 31 | 24 | 28 | 56 | 99 | 48.0 |
| FTP-1 | 97 | 48 | 65 | 47 | 64 | 3 | 79 | 100 | 62.9 |
Table 1: UniVTAC success rates (%). Baseline values are reported in UniVTAC and SimpleTouch ( Chen et al., 2026a ; Yang et al., 2026a ) . Astra uses ten sampled scene seeds per task. V : vision; V+T : vision and tactile RGB. Avg. is the unweighted mean over tasks. Bold denotes the best value in each column, including ties.
| Model | Weight Class. | Hardness Class. | Roughness Class. | Roughness Regrasp | Empty-Can Selection | Grasp Chip | Gear Assembly | Pull Drawer | Insert USB | Insert Block v1 | Insert Block v2 | Avg. |
| StarVLA (Qwen3) | 44 | 38 | 42 | 35 | 32 | 10 | 26 | 35 | 4 | 0 | 0 | 24.2 |
| StarVLA (Cosmos) | 51 | 67 | 94 | 9 | 13 | 1 | 33 | 22 | 3 | 1 | 3.3 | 27.0 |
| Lingbot-VA | 50 | 44 | 39 | 40 | 44 | 22 | 59 | 15 | 2 | 2 | 6 | 29.4 |
| FastWAM | 49 | 84 | 67 | 7 | 9 | 50 | 51 | 88 | 6 | 0 | 0 | 37.4 |
| XVLA | 28 | 30 | 56 | 47 | 45 | 42 | 84 | 76 | 2 | 3 | 0.7 | 37.6 |
| Xiaomi-Robotics | 52 | 46 | 54 | 62 | 38 | 24 | 73 | 84 | 5 | 2 | 0 | 40.0 |
Table 2: OpenViTac success rates (%). Baseline values are reported in OpenViTac ( Wu et al., 2026 ) . Astra uses ten sampled scene seeds per task, except Insert Block v2 (thirty across three shapes). V : vision; V+T : vision and tactile RGB. Avg. is the unweighted mean over tasks. Bold denotes the best value in each column, including ties. Class. abbreviates Classify; Roughness Regrasp abbreviates Roughness-Guided Regrasp. Grasp Chip follows the adjusted criterion described above.
Figure 3: Per-task changes with tactile input. Circles denote V , diamonds denote V+T , and the right-hand numbers show V+T minus V in percentage points. Six tasks improve, nine are unchanged, and four decrease.
Figure 4: Pull-out Key: resistance-aware extraction. Seed 1000017, V+T , success. At step 60, the action note identifies torsional deformation and proposes unloading the twist. Continued axial resistance leads to reseating and finer angular adjustment at step 80; the later frame shows extraction. The sequence alternates extraction attempts with unloading and realignment as resistance persists. Full rollout .
Figure 5: Insert Hole: inferring the bore direction. Seed 1000028, V+T , success. Astra interprets the opposing slopes of the tactile bands together with visible shaft deflection to infer a direction different from the reference trajectory. Astra pivots the shaft while maintaining entrance contact, then advances along the inferred bore. The case illustrates adaptation to contact geometry rather than literal replay of the demonstrated motion. Full rollout .
Figure 6: Hardness Classify: probing while maintaining retention. Seed 10006, V+T , success. Astra slightly relaxes the grasp after lifting, detects slipping, and re-establishes contact. The subsequent action note associates the sharply defined loaded edge with the hard-object demonstration. The probe changes the contact condition, and the subsequent correction preserves retention before classification. Full rollout .
Figure 7: Grasp Chip: cautious convergence on a fragile grasp. Seed 10099, V+T , success. Unilateral contact triggers smaller closure increments and a commanded 0.5 mm lateral correction at step 110, between panels (a) and (b). After bilateral contact develops, Astra avoids continued tightening and tests retention through a gentle lift. The episode later completes intact placement; the panels focus on the contact-establishment phase. Full rollout .
Figure 8: Gear Assembly: resolving successive assembly constraints. Seed 10034, V+T , success. When the red gear tilts, Astra unloads contact and adjusts its center. A later yaw adjustment addresses possible tooth interference before seating and rotation. The correction strategy changes from centering to tooth engagement as the observed constraint changes. Full rollout .
Figure 9: Insert USB: verifying object progress. Seed 10066, V+T , success. The connector remains exposed while the tactile contacts shift, and the action note identifies finger slip along a blocked plug. Astra withdraws, recenters, and issues a submillimeter correction before insertion progresses. Hand motion is not treated as sufficient evidence that the connector has entered the socket. Full rollout .
Figure 10: Insert Block v1: a local correction after rim contact. Seed 10036, cube, V+T , success. Astra prepares the hand orientation before grasping, transports the cube, and then responds to possible rim contact with a small retreat and a commanded 2 mm centering adjustment. The final frame shows the release-and-verification phase. The local retreat creates clearance for recentering before renewed insertion. Full rollout .
Figure 11: Insert Block v2: recovering from unsuccessful placement. Seed 10019, half-cylinder, V+T , success. After release, the part remains at the opening. Astra regrasps, lifts, and makes further alignment attempts until the part settles into the opening. The behavior demonstrates outcome verification and recovery from an incomplete placement. Full rollout .
Figure 12: Visible cues in nominally tactile tasks. Three V episodes from tasks with 100% sampled success. Left: a recorded camera view; center: enlargement of the green-marked region; right: the visible cue. Grasp Classify shows cylinder texture, Roughness Classify exposes surface ridges and contour structure, and Roughness-Guided Regrasp contains visually different surfaces even in the fixed-camera view. Enlargements preserve the recorded pixels. Rollouts: Grasp Classify , Roughness Classify , and Roughness-Guided Regrasp .
Figure 13: Limited sensitivity to tangential motion and load. Weight Classify, seed 10038, V+T , failure. Astra interprets small tactile changes during a lift as evidence of a light object and places the heavy block on the incorrect blue tray. The image pattern also depends on grasp position, aperture, and motion; a small shear response is therefore not an unambiguous weight measurement. Full rollout .
Figure 14: Recognizing misalignment does not ensure effective correction. Insert Block v1, seed 10016, V+T , failure. Repeated rim contact prompts regrasping and leveling. From step 240 onward, the cumulative commanded pitch increments total approximately , with compensatory translation, yet insertion remains incomplete at the action limit. Full rollout .
Figure 15: Recognizing obstruction without resolving the correction direction. Insert HDMI, seed 1000062, V+T , failure. Fixed-camera crops and paired tactile images show three stages of the same unsuccessful episode. The recorded notes identify pitch-sign uncertainty at action 235, a return toward an upright interpretation at 270, and persistent obstruction at 585. The changing contact patterns do not lead to successful insertion. Full rollout .
Figure 16: Repeated probing without timely convergence. Empty-Can Selection, seed 10034, V+T , failure. Astra compares load responses, changes the grasp after unstable contact, and repeats the comparison. It selects the correct blue can at step 370 and starts the final grasp-and-lift sequence at step 390, but exhausts the 400-action budget before transfer. Information gathering must be coordinated with a stopping rule and the remaining manipulation. Full rollout .
| Input | Input tokens | Cached input | Output tokens | Total tokens | CNY |
| V | 127,250,755 | 104,837,109 | 2,283,555 | 129,534,310 | 3,102.25 |
| V+T | 203,911,092 | 170,693,217 | 2,521,218 | 206,432,310 | 4,402.80 |
| Total | 331,161,847 | 275,530,326 | 4,804,773 | 335,966,620 | 7,505.05 |
Table 3: Evaluation-phase token usage and cost. V : vision; V+T : vision and tactile RGB. Phase totals include billed errors, retries, and reruns, but exclude demonstration preparation and later supplementary experiments. Cached input is included in input tokens; reasoning tokens are included in output tokens. Costs sum the provider’s logged estimates.
Figure 17: Distribution of API response latency. Five-second bins, normalized by the total number of valid responses in each setting: 3,927 for V and 4,890 for V+T . The plotted range ends at 120 s; the omitted tails remain in the normalization. Failed requests and responses without output tokens are excluded from latency statistics.
| Input | Requests | Mean | Median | P90 | P95 | P99 |
| V | 3,927 | 22.8 | 15.4 | 46.3 | 59.1 | 110.7 |
| V+T | 4,890 | 21.1 | 14.0 | 44.0 | 56.8 | 96.6 |
Table 4: Complete API response latency in seconds. V : vision; V+T : vision and tactile RGB. Each row summarizes valid, deduplicated responses with positive output tokens. P90, P95, and P99 denote the 90th, 95th, and 99th percentiles. Mean and quantiles include the full distribution, including responses beyond the displayed histogram range.
Appendix figures & tables3 assets
Supplementary material from the paper’s appendix.
Appendix
| Task | Shots | Segments | Total actions | Cameras | |
| UniVTAC | |||||
| Lift Bottle | 1 | 12 | 10 | 120 | Fixed |
| Pull-out Key | 1 | 12 | 5 | 60 | Fixed |
| Lift Can | 1 | 12 | 5 | 60 | Fixed + wrist |
| Put Bottle | 1 | 12 | 10 | 120 | Fixed |
| Insert Hole | 1 | 12 | 5 | 60 | Fixed |
Table 5: Demonstrations, segment lengths, and camera inputs. “Shots” counts distinct demonstration trajectories; “Segments” is the number selected per demonstration. is the number of actions per segment. Total actions counts demonstration actions in context, not an evaluation budget. The camera column lists official scene views; the visuo-tactile setting additionally receives both tactile RGB views. Insert Block v2 supplies all three shape demonstrations to each variant.
Figure 18: Paired outcomes, part I. Pull-out Key (seed 1000025), Lift Can (1000093), and Insert Hole (1000014). The V episodes fail and the corresponding V+T episodes succeed. These endpoint views complement the successful contact-inference sequences in Figures 4 and 5 .
Figure 19: Paired outcomes, part II. Insert HDMI (seed 1000063), Hardness Classify (10003), and Insert Block v1 (10008). The V+T episodes succeed where the same-seed V episodes fail. Hardness Classify requires the correct category-specific destination; the insertion tasks require the part to enter its matching opening. Complete action sequences and input views remain available in the online archive.
More in this topic
Robotic Manipulation
Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.
See all 1,671 papers