Agricultural tasks such as harvesting and pruning require robots to reach through dense foliage, where contact is often unavoidable and vision is obstructed. Intermittent contact creates an imbalance between free-space and contact-rich observations, complicating the learning of contact-responsive behaviours. We present SPICE, a tactileconditioned diffusion policy for reaching targets in dense plant canopies using tactile sensing as the sole environmental feedback. SPICE exhibits three behaviours: moving through compliant foliage, manoeuvring around partially obstructing stems, and manipulating branches that block the target path. To the best of our knowledge, SPICE is the first tactile-conditioned diffusion policy for target reaching through physically cluttered plant canopies. Across over 180 real-robot trials, SPICE was evaluated against tactile-conditioned, image-conditioned, and reactive-control baselines. Spatial tactile information and explicit contact conditioning proved important, with SPICE achieving 95% branch-manoeuvring success versus 45% for the strongest tactile baseline. SPICE also achieved branch manipulation beyond the reactive controller and remained effective under visual occlusion, where the image-conditioned DP failed. Zero-shot transfer across 40 trials on unseen tomato and lime plants achieved 87.5% success rate. These results demonstrate the potential of tactile-conditioned generative policies for interaction in visually occluded plant environments.
Figures & tables
Fig. 1: Contact-responsive motion adaptation of SPICE in a cluttered tomato canopy. The blue inset shows tactile-driven trajectory adaptation, with the red and gray dashed lines indicating the modified and original paths, respectively. Tactile sensing enables SPICE to adapt its motion. The bottom row shows the interaction sequence from the tomato plant experiment: the end effector contacts the branch ( t1 ), pushes it downward ( t2 ), moves above the displaced branch ( t3 ), and continues toward the target ( t4 ).
Fig. 2: Contact-dependent behaviours for reaching an occluded target (red): direct motion towards target through compliant foliage, manoeuvring around an obstructing branch, and displacing a branch away from the target path. Blue arrows indicate the end-effector trajectories.
Fig. 3: SPICE is a tactile-conditioned diffusion policy in which a CNN encodes the three-axis tactile tensor, which is combined with explicit contact conditioning and the current and target robot configurations to condition joint-space action generation.
Fig. 4: Mock branch setup for demonstration collection and evaluation under visual occlusion, and the GELLO teleoperation setup with tactile heat-map feedback.
Fig. 5: Distributions of end-effector positions and interaction behaviours in the demonstration dataset.
Fig. 6: Comparison of End Effector (EE) trajectories of SPICE with aggregated, direct-taxel, and hand-crafted tactile representations in vertical-branch trials. Representative trajectories show contact responses from a matched initial condition. Box plots report target error, action discontinuity, Peak Tactile, and Ryx .
Policy
Task success
Target error (cm)
Discontinuity ( ∘ /replan)
Peak Tactile (sensor units)
Ryx during contact
Fxyz -DP +ct
0/10 (0%)
6.68±3.69
3.77±1.84
2097±2300.42
0.62±0.25
Fxyz -DP w/o ct
0/10 (0%)
8.92±0.31
5.83±1.54
7150.11±932.88
0.32±0.20
RICE-y-DP +ct
8/20 (40%)
8.20±5.12
1.93±1.11
3704.65±3509.58
0.62±0.49
RICE-DP +ct
3/20 (15%)
8.64±4.81
2.08±0.85
4574.69±2098.38
0.44±0.26
TactDP w/o ct
7/20 (35%)
8.40±4.39
4.08±1.87
3853.03±3397.36
0.58±0.46
SPICE w/o ct
0/10 (0%)
8.48±4.89
1.62±0.42
4909.29±3382.96
0.48±0.18
TABLE I: Expt. I. Tactile representation
Fig. 7: Tactile-encoder ablation comparing MLP, Transformer, and CNN encoders. Box plots report target error and peak sustained tactile response for horizontal and vertical trials, action-chunk discontinuity during contact, the vertical lateral-to-forward displacement ratio, and horizontal end-effector pitch change and branch displacement. Lower discontinuity indicates smoother transitions between action chunks.
Fig. 8: Comparison of SPICE and RICE. Box plots show target error, vertical tactile-gradient magnitude, peak sustained tactile response, Ryx , horizontal pitch change, and downward branch displacement. RICE followed oblique vertical branches, whereas SPICE reduced contact and displaced horizontal branches from the target path.
Fig. 9: comparison of SPICE with RICE. (a) Robot configuration before and after contact in vertical-branch trial 9, where SPICE manoeuvres around the branch. (b) End-effector and branch trajectories for SPICE and RICE in trial 9. SPICE moves around the branch, whereas RICE initially moves along it before passing around it. (c) Robot configuration before and after contact in horizontal-branch trial 61, where SPICE pushes the branch downward before proceeding toward the target. (d) End-effector and branch trajectories for SPICE and RICE in trial 61. SPICE displaces the branch from its path, whereas RICE does not. (e) Box plot comparison of ImageDP and SPICE. (f) EE trajectory lime and tomato real plant experiment.
Task
Metric
RICE
SPICE
Branch manoeuvring
Task success
30/30 (100%)
30/30 (100%)
Tactile gradient (sensor unit/m, ×104 )
10.03±7.73
3.32±3.27
Peak tactile response (sensor units, ×103 )
18.53±13.53
12.72±11.64
Ryx
0.87±0.54
0.67±0.52
Branch manipulation
Task success
0/30 (0%)
20/20 (100%)
Δ EE pitch ( ∘ )
0.05±0.05
7.95±5.45
TABLE II: Exp III: SPICE vs RICE
Metric
Image DP
SPICE
Target error (cm)
3.44±1.43
1.97±1.28
Action discontinuity ( ∘ )
1.24±0.52
2.77±1.12
Reach target success (no branch)
13/20
19/20
Branch manoeuvre task success
0/20
18/20
TABLE III: Exp. IV: Visual occlusion (SPICE vs. Image DP)
Fig. 10: Image sequence of SPICE manoeuvring around a lime branch.
Fig. 11: Box plot comparison for lime and tomato experiments
Imitation learning has achieved impressive results in robotic manipulation, yet most existing approaches assume clean backgrounds and lack explicit mechanisms for obstacle-aware motion generation. Extending such policies to cluttered, real-world scenes with unstructured obstacles remains a key generalization challenge. We present ObstaDiff, a decomposed diffusion-policy framework with a lightweight obstacle-aware visual encoder. ObstaDiff extracts a structured target-obstacle-background representation, enabling the downstream alignment policy to generate end-effector trajectories toward a target-centered bottleneck pose while reasoning about surrounding obstacles. We evaluate ObstaDiff on 61 real-robot greenhouse trials per method (366 executions in total). ObstaDiff achieves 75.41% average task success and 8.20% average obstacle collision rate, outperforming representative imitation-learning baselines and improving generalization in cluttered agricultural scenes.
Jiawen Wang, Kevin Yao, Khalid Jawed
Department of Mechanical and Aerospace Engineering, University of California, Los Angeles · Department of Computer Science, University of California, Los Angeles
Visual policies learned from human videos, teleoperation, and robot demonstrations offer scalable motion priors, but often fail in contact-rich manipulation, where success significantly depends on local force and contact geometry. Tactile sensing provides these complementary signals, yet tactile data remain costly to collect and hard to generalize across sensors, robots, and tasks. We introduce OmniTacTune, a policy-agnostic real-world RL pipeline that adapts tactile feedback to pretrained visual policies through residual correction. OmniTacTune uses a two-stage design: it first bootstraps tactile-aware learning from autonomous base-policy rollouts, then learns a lightweight tactile residual policy through online interaction. Extensive experiments show that OmniTacTune generalizes across diverse contact-rich tasks, visual base policies, and tactile representations. Across four real-world contact-rich tasks, it improves visual base policies from 5-40% success to 85-100% within 40-80 minutes, demonstrating an efficient path for adapting tactile feedback to scalable visual robot policies. Project page: https://colinyu1.github.io/omnitactune-site/
Kelin Yu, Haode Zhang, Harish Ravichandar +2
University of Maryland, College Park · Georgia Institute of Technology
General-purpose multimodal models have shown promise as direct robot action policies, yet whether they grasp the physics underlying real-world interactions and can apply that understanding to guide action remains underexplored. We investigate this question using contact-rich manipulation as a representative testbed, examining tactile understanding as a concrete instance of this broader capability. Contact-rich manipulation requires interpreting tactile feedback and adjusting actions after obstruction. Existing learned tactile policies acquire these capabilities through specialized pretraining and per-task optimization, with substantial data and training costs. We present GPT-as-Tactile-Policy, which summarizes expert demonstrations into task guidance and tactile references for contact assessment and corrective action. GPT-6-Astra combines this context with multimodal observations and interaction history to generate control commands and adjust actions through contact feedback. Without additional robotics-specific pretraining or per-task policy fine-tuning, our method achieves 93.8% mean success across five UniVTAC tasks. Overall performance is comparable to state-of-the-art policies, with higher success on selected tasks. On Insert Hole, tactile feedback improves success from 31% to 99%. Execution analysis further demonstrates contact-guided probing and correction, supporting manipulation progress through explicit expert experience and online feedback.
Ziyong Wang, Yupei Lin, Runxuan Huang +3
Sun Yat-sen University · X-Era AI Lab · The Hong Kong Polytechnic University