SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

Chen Yang; Linzhe Shi; Changjie Wu; Hang Zhang; Ronghan Chen; Lingjun Zhang; Xu Hu; Mu Xu

cs.ROOct 2, 2026

SimpleTouch: Can Vision-Language-Action Models Master Contact-Rich Manipulation Without Tactile Policy Pretraining?

Authors: Chen Yang, Linzhe Shi, Changjie Wu, Hang Zhang, Ronghan Chen, Lingjun Zhang, Xu Hu, Mu Xu, +2 more

Organizations: Tsinghua University · Amap, Alibaba Group · Hong Kong Polytechnic University

Abstract

Tactile sensing provides essential contact information for robotic manipulation, yet incorporating it into pretrained vision-language-action (VLA) models remains challenging. A common concern is that simply introducing touch during task-specific fine-tuning may fail to bridge the cross-modal gap, yielding limited gains or even reduced success. Consequently, existing methods often rely on large-scale tactile policy pretraining or separate visuotactile alignment, adding data requirements and training stages. We introduce SimpleTouch, a simple VLA extension that augments $π_{0.5}$ with a tactile expert, to test whether these additional stages are necessary. Leveraging all tokens from a frozen pretrained tactile encoder, the expert learns from action supervision and multi-horizon prediction of future tactile latents. This single-stage training uses only task demonstrations, without additional tactile policy pretraining or separate alignment. With 50 demonstrations per task, SimpleTouch achieves the highest success rate among evaluated methods on all six UniVTAC tasks. Its average success rate reaches 77.5%, compared with 45.2% for FTP- $π_{0.5}$ and 66.7% for FTP-1, corresponding to gains of 32.3 and 10.8 percentage points, respectively. Across four real-world tasks, it averages 71.3%, exceeding FTP-1 by 8.8 percentage points. These results demonstrate that, given pretrained VLA and tactile representations, additional tactile policy pretraining is not a prerequisite for strong performance on these tasks, offering a simpler route to contact-rich manipulation. Project page: https://simpletouch-robot.github.io/

Figures & tables

Figure 1: Overview of \simpletouchfont SimpleTouch. \simpletouchfont SimpleTouch combines an off-the-shelf pretrained tactile encoder with a tactile expert to learn from 50 task-specific demonstrations without tactile policy pretraining. The chart on the right reports success rates on six UniVTAC and four real-robot tasks.

Figure 2: \simpletouchfont SimpleTouch architecture. A tactile expert uses complete current tactile features from a frozen encoder and learns multi-horizon future prediction. In the right-side training and inference masks, rows are queries, columns are keys/values, and colored cells mark permitted attention. The symbols $c_{0}$ , $z_{0}$ , $z_{1:H}$ , and $a_{1:H}$ denote the current visual-language and state context, current tactile tokens, future tactile tokens, and action tokens, respectively, following the token order shown above.

Figure 3: Real-world setup and rollouts. The left panel shows the PiPER platform and sensing hardware; the right panel shows all four tasks—Play Mahjong, Wipe Board, Pick Up Chips, and Insert USB—with time progressing from left to right throughout each rollout.

Method	Lift Bottle	Pull-out Key	Lift Can	Put Bottle	Insert Hole	Insert Tube	Avg.	Avg. w/o Lifts
ACT	42	28	20	28	19	45	30.3	30.0
$\pi_{0.5}$	92	36	59	31	26	34	46.3	31.8
VITaL	72	47	8	32	25	34	36.3	34.5
UniVTAC-ACT	71	46	29	31	24	56	42.8	39.3
Tactile-VLA	97	32	15	10	41	56	41.8	34.8
FTP- $\pi_{0.5}$	77	30	26	19	47	72	45.2	42.0

Table 1: Results on the UniVTAC simulation benchmark. We report success rates (%). “Avg. w/o Lifts” excludes Lift Bottle and Lift Can. We highlight the best and second-best results.

Method	Play Mahjong	Wipe Board	Pick Up Chips	Insert USB	Avg.
ACT	45	15	30	0	22.5
$\pi_{0.5}$	40	45	55	10	37.5
UniVTAC-ACT	70	35	25	0	32.5
FTP-1	95	65	60	30	62.5
\simpletouchfont SimpleTouch	100	85	70	30	71.3

Table 2: Results on four real-world contact-rich manipulation tasks. We report success rates (%). We highlight the best and second-best results.

Panel of Figure 4 — Figure 4: (Left) Effect of the number of retained current tactile tokens per sensor; fewer tokens imply stronger spatial pooling, 0 is vision-only, 1 retains only the T3 CLS token, and 197 is the full T3 sequence. (Right) Effects of tactile-encoder variants across tasks; VAE uses a frozen visual VAE, while Trainable T3 fine-tunes T3.

Variant	Pull-out	Put	Insert	Insert	Avg.
	Key	Bottle	Hole	Tube
Vision-only	36	31	25	64	39.0
w/o Tac. Pred.	60	37	83	95	68.8
Single-horizon	53	44	79	98	68.5
\simpletouchfont SimpleTouch	62	50	85	100	74.3

Table 3: Future tactile prediction ablations on UniVTAC. We highlight the best and second-best values.

Figure 6: Data efficiency on UniVTAC. Success rates of FTP-

\pi_{0.5}

and \simpletouchfont SimpleTouch on Put Bottle and Insert Hole across demonstration budgets.

Design choice	FTP- $\pi_{0.5}$	\simpletouchfont SimpleTouch
Tactile encoder	T3	T3
Encoder update	Fine-tuned	Frozen
Current tactile tokens	CLS only	All tokens
Tactile expert	Yes	Yes
Future tactile prediction	No	Multi-horizon

Table 4: Tactile design choices in FTP-

\pi_{0.5}

and \simpletouchfont SimpleTouch; shaded rows mark design choices that differ between the two methods.

Figure 7: Training and inference efficiency of Joint, IDM, and Fast. The left panel shows success rate versus inference latency; dashed lines connect formulations evaluated on the same task. The right panels show mean training time (top) and mean task-wise success-rate/latency ratio (bottom).

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Value
Vision-language-action backbone
Base policy	$\pi_{0.5}$
Visual-language backbone	PaliGemma ( Beyer et al., 2024 ) : SigLIP ( Zhai et al., 2023 ) vision encoder and Gemma 2B language model
Action expert	Gemma 300M; width 1024; 18 Transformer layers ( Vaswani et al., 2017 )
State and action spaces	Shared 8-dimensional interface: UniVTAC uses seven arm joints and one gripper coordinate; PiPER uses six arm joints, one zero dummy joint, and one gripper coordinate. States are absolute; action-joint dimensions are relative and the gripper action is absolute. Both are zero-padded to 32 internal model dimensions
Action chunk length	50

Appendix

Table 5: Model, training, and inference configurations for \simpletouchfont SimpleTouch.

Task	Demonstrations	RGB views	Predicted	Executed	Eval. trials
Lift Bottle	50	Head	50	50	100
Pull-out Key	50	Head	50	50	100
Lift Can	50	Head + wrist	50	50	100
Put Bottle	50	Head	50	50	100
Insert Hole	50	Head	50	25	100
Insert Tube	50	Head + wrist	50	25	100

Appendix

Table 6: UniVTAC task protocol for \simpletouchfont SimpleTouch. “Predicted” and “Executed” denote the action-prediction and execution horizons, respectively.

Figure 8: UniVTAC simulation task rollouts. Each row corresponds to one of the six evaluated tasks and shows five key frames illustrating its manipulation and contact progression ( Chen et al., 2026a ; Yuan et al., 2026a ) .

Figure 9: Hardware used for real-robot experiments. (a) PiPER six-degree-of-freedom arm. (b) Pika parallel gripper and teleoperation interface. (c) GelSight Mini tactile sensor; two units are mounted on opposing gripper fingers. Product images are from the respective manufacturers.

Figure 10: Two-part 3D-printed mount for the GelSight Mini. (a) Main bracket and (b) companion base; the columns show front, side, top, and perspective views. Both parts form one gripper-mounted sensor assembly.

Task	Demonstrations	RGB view	Predicted	Executed	Eval. trials
Play Mahjong	50	Fixed	50	50	20
Wipe Board	50	Wrist	50	50	20
Pick Up Chips	50	Fixed	50	25	20
Insert USB	50	Fixed	50	50	20

Appendix

Table 7: Real-robot execution protocol for \simpletouchfont SimpleTouch. “Predicted” and “Executed” denote the action-prediction and execution horizons, respectively.

Task	RGB view	Initial-condition variation	Binary success criterion
Play Mahjong	Fixed	The grasped tile is either a bird tile or a one-dot tile; its pose within the gripper is varied. A two-second tactile observation is recorded at the start.	The bird tile is discarded to the table center, or the one-dot tile is placed in front of the robot, followed by a stable release.
Wipe Board	Wrist	Board pose and angle, writing size, and eraser placement in the gripper are varied.	The marked region is wiped clean while the eraser remains securely grasped.
Pick Up Chips	Fixed	The chip and supporting bottle-cap positions are varied.	The chip is grasped and lifted clear without being dropped or visibly broken.
Insert USB	Fixed	USB placement in the gripper is varied while its insertion orientation is kept consistent with the port.	The connector is fully seated in the port and remains inserted after release.

Appendix

Table 8: Real-robot task protocols. Each policy is trained with 50 demonstrations and evaluated for 20 rollouts per task.

Variant	Controlled change	Exact setting
Vision-only	Remove touch from the policy	Disable the tactile encoder and tactile expert while retaining the task data and $\pi_{0.5}$ action backbone.
Token count	Vary current-touch spatial resolution	Use the T3 CLS token alone (1), or CLS plus adaptively pooled $3{\times}3$ (10), $6{\times}6$ (37), or $8{\times}8$ (65) spatial tokens; 197 retains CLS plus the complete $14{\times}14$ grid. Counts are per sensor.
VAE	Replace the tactile-specific representation	Use a frozen Wan2.2 video VAE from Fast-WAM. The two RGB tactile images are concatenated, encoded as one latent frame, and rearranged into 98 tokens of width 192 before projection to the tactile expert. No CLS token or spatial pooling is used.
Trainable T3	Update the tactile encoder	Optimize T3 jointly with the policy from current tactile images; keep future-prediction targets fixed to cached features from the original pretrained T3.
w/o Tac. Pred.	Remove predictive supervision	Set the future-tactile loss weight to zero while retaining the full current-touch pathway and tactile expert.
Single-horizon	Use one future prediction target	Predict CLS plus the same pooled $6{\times}6$ spatial grid (37 tokens per sensor) only at $t{+}50$ , rather than at ten offsets $t{+}5,\ldots,t{+}50$ .

Appendix

Table 9: Definitions of the reported representation and prediction ablations.

Mode	Training-time action conditioning	Inference
Joint	Action tokens read the K/V cache produced by the same noisy future-tactile branch. Action and tactile flow times are sampled independently, and future-tactile queries never read action tokens.	Future touch and actions are updated together for 10 Euler steps.
IDM	Actions read a separate teacher-forced future-tactile branch. Each sample uses the clean target or, with probability 0.5, an independently noised target; this branch is separate from the auxiliary future-loss branch.	Future touch is denoised for 10 steps, cached at its clean endpoint, and then conditions 10 action-denoising steps.
Fast	The action branch reads only visual-language, state, and current-tactile context. Future tactile prediction is an auxiliary training objective and cannot condition action tokens.	The cached current context directly conditions 10 action-denoising steps; no future tactile latent is generated.

Appendix

Table 10: Training and inference protocols for the three tactile-prediction formulations.

Figure 11: Additional multi-horizon future-tactile predictions for Lift Bottle, Pull-out Key, Lift Can, and Insert Tube. For each task, the current tactile image and its T3 representations appear on the left; subsequent columns compare the observed tactile image, predicted pooled latent, and ground-truth pooled latent at 5-step offsets through $t+50$ .

Execution	Lift Bottle	Pull-out Key	Lift Can	Put Bottle	Insert Hole	Insert Tube	Avg.
H50	100	62	68	50	58	88	71.0
H25	100	52	42	39	85	100	69.7

Appendix

Table 11: Effect of execution horizon. Every policy is trained with a 50-action prediction horizon; H25 and H50 execute 25 and 50 actions, respectively, before the next observation. Success rates are reported in percent.

Task	Formulation	Training (s/step)	Latency (s/query)	SR (%)	SR/latency (%/s)
Pull-out Key	Joint	1.891	0.946	56	59.2
	IDM	2.275	1.642	51	31.1
	Fast	1.862	0.614	62	101.0
Put Bottle	Joint	1.890	1.058	46	43.5
	IDM	2.279	1.444	38	26.3
	Fast	1.856	0.632	50	79.1

Appendix

Table 12: Task-wise training and inference efficiency for Joint, IDM, and Fast.

Explore similar work

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation
Sep 17, 2026Zhengyu Tao, Xin Li, Xin WangContact-Rich Robotic Manipulation Vision-Language-Action Models
Texas A&M University, College Station, TX 77843, USA
Tactile Modality Fusion for Vision-Language-Action Models
Mar 15, 2026Charlotte Morissette, Amin Abyaneh, Wei-Di Chang +5Contact-Rich Robotic Manipulation Vision-Language-Action Models
McGill University, CAN · Mila - Québec AI Institute, CAN · NVIDIA, USA
TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models
Jun 27, 2026Mark Van der Merwe, Mohamad Louai Shehab, Jayjun Lee +4Vision-Language-Action Models Tactile Sensing for Robotic Manipulation
Robotics Department, University of Michigan · Computer Science and Engineering Department, University of Michigan

Sep 17, 2026, cs.ROJ/K move · Enter open · S save

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation

Zhengyu Tao, Xin Li, Xin Wang

Texas A&M University, College Station, TX 77843, USA

Contact-Rich Robotic Manipulation Vision-Language-Action Models VLMs for Robotics Tactile Sensing for Robotic Manipulation

Open paper AskPDF arXiv

Vision-language-action (VLA) models have demonstrated strong capabilities in robotic manipulation, yet their reliance on visual perception limits robustness in contact-rich environments, where critical physical interaction states may not be visually observable. Existing tactile-enhanced VLA methods improve physical grounding using observed tactile feedback, but most remain largely reactive rather than explicitly modeling how contact may evolve. Therefore, we propose ForeTac-VLA, a forecasting-based tactile-vision-language fusion model that predicts future tactile states to guide action generation. Specifically, ForeTac-VLA encodes recent tactile observations into temporal representations and integrates them with vision-language features through bidirectional cross-attention. Further, a transformer-based forecasting module predicts multi-step future tactile states, enabling the model to reason jointly over observed and anticipated contact. Finally, the fused multimodal representations and predicted future tactile states are fed into the VLA backbone to condition action generation. To stabilize training, a ground-truth-to-prediction curriculum is employed when early forecasts are unreliable. Across four real-world contact-rich manipulation tasks, ForeTac-VLA achieves an average success rate of 95%, outperforming the fine-tuned VLA model by 36.25 percentage points and state-of-the-art tactile-enhanced VLA baselines by over 22 percentage points. ForeTac-VLA also maintains strong performance under low-illumination and visually cluttered conditions. Video demonstrations can be found on https://foretac-vla.github.io/

Abstract

Explore similar work

ForeTac-VLA: A Forecasting-Based Tactile-Vision-Language-Action Model for Contact-Rich Robotic Manipulation

Tactile Modality Fusion for Vision-Language-Action Models

TAP-VLA: Tactile Annotation Prompting for Vision Language Action Models