PCTac: Mechanochromic Photonic-Crystal Tactile Sensor for Dexterous Manipulation
Organizations: School of Information Science and Technology, ShanghaiTech University, Shanghai, China. · State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China.
Abstract
Modern visuotactile sensors primarily sense contact by imaging the deformation of a compliant interface. Using techniques such as photometric stereo, they can recover detailed contact geometry, often represented as surface normals or depth. However, force-related information typically still needs to be inferred indirectly through calibration, geometric reconstruction, or learned mappings. Mechanochromic photonic-crystal (PC) materials offer an alternative physical encoding mechanism: contact loading changes their periodic microstructure and spectral response, directly transforming local stress into a spatially resolved structural-color field. Despite this potential, PC materials have rarely been explored as visuotactile sensors for closed-loop robot learning. We present PCTac, an open-source mechanochromic PC fingertip that uses a dense hue field as its tactile observation. Loading characterization demonstrates a repeatable relationship between the applied normal load and the observed hue response. We further integrate this representation directly into a Diffusion Policy and evaluate it on three real-robot manipulation tasks. Across 90 physical rollouts, PCTac Hue achieves 68 successes (75.6%), compared with 60 (66.7%) for PCTac RGB, 57 (63.3%) for Wedge RGB, and 29 (32.2%) for vision-only. PCTac Hue also achieves the highest observed success rate on each evaluated task, including Perturbed Test-Tube Insertion, where it achieves 16/30 successes versus 13/30 for tactile RGB baselines and 1/30 for vision-only. These results demonstrate the potential of mechanochromic structural color as a compact, load-sensitive tactile representation for robot learning and closed-loop manipulation. Project page: https://bey0ndt1me.github.io/PCTac-page/.
Figures & tables
| Sensor | Signal encoding | Illumination colors | Direct stress readout | Dense field | Learning use |
| GelSight family † [ 1 , 2 , 3 , 4 ] | Geometry / displacement | R, G, B | |||
| TacTip † [ 21 , 33 ] | Displacement | White | |||
| FingerVision [ 34 ] | Displacement | Ambient | |||
| 9DTact † [ 35 ] | Deformation | White | |||
| GelStereo [ 36 ] | Geometry | White | |||
| Sasso2026 [ 17 ] | Stress | White |
| Policy | Tactile representation | Whiteboard Wiping ( ) | Test-Tube Insertion ( ) | Perturbed Insertion ( ) | Overall ( ) |
|---|---|---|---|---|---|
| Vision-only | None | 19/30 (63.3%) | 9/30 (30.0%) | 1/30 (3.3%) | 29/90 (32.2%) |
| Wedge RGB | Raw RGB | 23/30 (76.7%) | 21/30 (70.0%) | 13/30 (43.3%) | 57/90 (63.3%) |
| PCTac RGB | Raw RGB | 22/30 (73.3%) | 25/30 (83.3%) | 13/30 (43.3%) | 60/90 (66.7%) |
| PCTac Hue | Circular hue difference | 24/30 (80.0%) | 28/30 (93.3%) | 16/30 (53.3%) | 68/90 (75.6%) |
Tactile Sensing
Tactile sensing measures contact forces, deformation, texture, and motion to help robots perceive and act through physical interaction.
See all 152 papers