Large-scale representation learning has produced public foundation models that provide strong general-purpose visual features. However, their pretraining objectives are not designed for robot representation learning, so the resulting embeddings tend to emphasize global semantics rather than the temporally sensitive representations required for robot control. Training robot models from scratch is also undesirable due to the limited scale and high collection cost of robotics data, as well as the substantial computational burden of pre-training. In this paper, we propose SPARK (Simple Post-training for Adapting pRetrained Knowledge), a post-training method that adapts foundation models to robot control. SPARK is built on two principles: temporal abstraction, which encourages the encoder to derive compact features that preserve information essential for understanding temporal changes and action-relevant scene structure, and knowledge preservation, which aligns patch-level representations with those of the original foundation model to retain useful pretrained semantics. These objectives yield compact visual state representations that remain semantically meaningful while preserving useful prior knowledge. Experiments across multiple robotics benchmarks show that SPARK consistently improves success rates and generalization over vanilla foundation models, and further demonstrate that these gains transfer to real-world robot manipulation.
Figures & tables
Figure 1: Overview of SPARK (Simple Post-training for Adapting pRetrained Knowledge). SPARK encourages temporal abstraction of patch-wise semantics while preserving the foundation model’s pre-trained knowledge. Given a reference scene xt , the SPARK encoder fθ produces a class token as a compact abstraction of xt , from which the decoder predicts target token representations with hints from a few target patches. The Temporal Abstraction Loss LTA aligns these predictions with the target representations of a frozen foundation model fθ0 , while the Knowledge Preservation Loss LKP aligns the target patch tokens of fθ and fθ0 . This encourages the class token to serve as a compact visual state for control while retaining the patch-level semantics of the original foundation model.
Method
MetaWorld
DMC
Franka Kitchen
Average
CLIP
Baseline
72.8 ± 1.1
47.9 ± 2.1
31.5 ± 2.4
50.7
SPARK
77.7 ± 7.0
50.6 ± 0.9
39.8 ± 5.6
56.0
Gain
+ 4.9
+ 2.7
+ 8.3
+ 5.3
SigLIP
Baseline
73.0 ± 7.6
42.1 ± 1.5
31.2 ± 0.6
48.8
SPARK
76.9 ± 6.8
42.6 ± 1.8
42.2 ± 2.4
53.9
Gain
+ 3.9
+ 0.5
+ 11.0
+ 5.1
Table 1: Experimental results on vision-based robot policy learning across MetaWorld, DMC, and Franka Kitchen. We report the average performance of imitation learning agents trained upon representations of CLIP [ 32 ] , DINOv2 [ 30 ] , SigLIP [ 42 ] , and Theia [ 34 ] post-trained on Kinetics-400 [ 21 ] dataset. Success rates (%) are reported for MetaWorld [ 40 ] and Franka Kitchen [ 13 ] , and normalized scores are reported for DMC [ 37 ] . We report the gains of our method over the baseline.
Figure 2: Real-world demonstration trajectories. Each row shows the temporal progression of a policy from the initial observation to intermediate interaction states and the final outcome. Across tasks, the SPARK-adapted policy exhibits stable approach, contact, and object manipulation behaviors, illustrating that the adapted visual representations capture control-relevant scene information that transfers reliably to real-world settings.
Figure 4
Methods
Architecture
Metaworld
Self-supervised Learning
R3M ∘
ViT-S
67.0
VC-1
ViT-B
86.4
data4robotics
ViT-B
87.0
Supervision with Aux. Language Guidance
R3M
ViT-S
69.2
Table 3: Comparative analysis of vision-based robot policy learning on MetaWorld. We report the performance of imitation learning agents on Metaworld [ 40 ] . The success rates (%) are reported for all the tasks.
Method
Success rate ( % )
Aligned VLM
Non-aligned VLM
SigLIP
39.2
29.5
SPARK (SigLIP)
41.4
31.2
Table 4: Comparative analysis of vision towers on VLA systems for long-horizon tasks. We consider two settings: an aligned VLM setting, where projector alignment is first conducted on the LLaVA-Pretrain dataset of 558K image-caption pairs while the non-aligned VLM setting skips this step. In both settings, the action expert is trained from scratch on LIBERO-10 while the full VLM remains frozen. Success rates are reported over 500 rollouts with 50 rollouts per task.
Image → Text
Text → Image
Model
R@1
R@5
R@10
R@1
R@5
R@10
SigLIP
90.4
98.6
99.0
75.6
93.2
96.3
SPARK (SigLIP)
87.6
97.7
98.8
74.0
92.4
95.6
Δ
−2.8
−0.9
−0.2
−1.6
−0.8
−0.7
Table 5: SPARK preserves vision-language alignment. Image-Text and Text-Image retrieval on Flickr30K 1K test [ 39 ] , which consists of 5,000 ground-truth pairs. We use the original SigLIP text encoder, so any drop is attributable to vision-side drift. Δ denotes the difference between the SPARK model and its baseline.
Variant
Knob1 on
Light on
Sdoor open
Ldoor open
Micro open
Average
SigLIP
17.0 ± 1.2
38.5 ± 1.9
75.5 ± 3.4
8.5 ± 1.0
16.5 ± 1.9
31.2 ± 0.6
LTA -only
5.5 ± 1.9
11.5 ± 5.3
43.0 ± 6.6
6.5 ± 1.0
6.5 ± 1.0
14.6 ± 2.5
LKP -only
12.0 ± 2.8
19.5 ± 1.9
67.0 ± 2.6
7.0 ± 1.2
8.5 ± 1.9
22.8 ± 0.7
SPARK
27.5 ± 5.0
43.0 ± 5.0
87.5 ± 1.9
20.0 ± 2.8
33.0 ± 5.3
42.2 ± 2.4
Table 6: Per-loss ablation. We ablate each loss individually using SigLIP on Franka Kitchen. The success rates (%) are reported for all the tasks.
Post-training
Franka Kitchen
Baseline
31.5
Masked autoencoding
23.0
Correspondence matching
25.5
SPARK
39.8
Table 7: Ablation study varying post-training recipes. We compared our SPARK to other post-training recipes on Franka Kitchen [ 13 ] using CLIP [ 32 ] as an initial model. The success rates (%) are reported for all the tasks.
Method
Masking strategy
Knob1 on
Light on
Sdoor open
Ldoor open
Micro open
Average
SigLIP
-
17.0 ± 1.2
38.5 ± 1.9
75.5 ± 3.4
8.5 ± 1.0
16.5 ± 1.9
31.2 ± 0.6
SPARK
Object-aware masking
8.5 ± 1.0
20.0 ± 3.3
60.0 ± 5.4
8.5 ± 1.0
10.0 ± 1.6
21.4 ± 1.2
Object-preserving masking
12.5 ± 2.5
20.5 ± 6.8
72.5 ± 4.4
9.5 ± 1.9
10.5 ± 1.0
25.1 ± 2.1
Random masking
27.5 ± 5.0
43.0 ± 5.0
87.5 ± 1.9
20.0 ± 2.8
33.0 ± 5.3
42.2 ± 2.4
Table 8: Ablation study on masking strategies. We replace random masking with structured variants based on the object-relevance criterion following AutoMAE [ 4 ] , using SigLIP on Franka Kitchen. The success rates (%) are reported for all the tasks.
Method
Dataset
Knob1 on
Light on
Sdoor open
Ldoor open
Micro open
Average
CLIP
-
23.0 ± 2.6
29.5 ± 4.7
69.5 ± 4.4
13.5 ± 3.4
22.0 ± 2.8
31.5 ± 2.4
SPARK
SSv2
28.5 ± 1.9
37.0 ± 6.8
78.0 ± 3.7
19.0 ± 4.2
15.0 ± 3.8
35.5 ± 1.1
Kinetics-400
31.5 ± 1.9
38.5 ± 6.0
74.5 ± 11.4
18.0 ± 2.0
29.0 ± 6.2
39.8 ± 5.6
Table 9: Generalizability across post-training datasets. We post-train CLIP on Something-Something-V2 (SSv2), an egocentric hand-object interaction dataset, with the same configuration as Kinetics-400. The success rates (%) are reported for all the tasks on Franka Kitchen.
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Figure A: Environments for evaluation. We visualize three vision-based robot learning benchmarks employed for the validation of our method: (a) Franka Kitchen [ 13 ] , (b) DeepMind Control Suite [ 37 ] , and (c) Metaworld [ 40 ] .
Method
Walker-stand
Walker-walk
Reacher-easy
Cheetah-run
Finger-spin
Average
CLIP
Baseline
58.1 ± 5.8
24.9 ± 2.8
75.2 ± 4.0
12.7 ± 2.0
68.4 ± 3.1
47.9 ± 2.1
SPARK
62.0 ± 3.5
29.5 ± 4.5
75.6 ± 4.7
22.0 ± 2.1
63.7 ± 1.6
50.6 ± 0.9
Gain
+ 3.9
+ 4.6
+ 0.4
+ 9.3
- 4.7
+ 2.7
SigLIP
Baseline
43.9 ± 5.9
19.9 ± 2.8
74.6 ± 2.5
9.0 ± 2.5
63.6 ± 0.4
42.1 ± 1.5
SPARK
45.2 ± 6.2
17.5 ± 2.4
75.2 ± 3.0
8.4 ± 2.1
66.8 ± 1.2
42.6 ± 1.8
Gain
+ 1.3
-2.4
+ 0.6
-0.6
+ 3.2
+ 0.5
Appendix
Table A: Experimental results on vision-based robot policy learning on DeepMind Control Suite (DMC). We report the performance of imitation learning agents on the DMC benchmark [ 37 ] , which are trained upon representations of CLIP [ 32 ] , DINOv2 [ 30 ] , SigLIP [ 42 ] , and Theia [ 34 ] post-trained on Kinetics-400 [ 21 ] dataset. The normalized scores are reported for all the tasks. We report the gains of our method over the baseline.
Method
assembly
bin-picking
button-press
drawer-open
hammer
Average
CLIP
Baseline
55.2 ± 6.6
63.2 ± 7.7
58.4 ± 13.4
100.0 ± 0.0
87.2 ± 15.6
72.8 ± 1.1
SPARK
72.0 ± 12.0
63.2 ± 7.0
58.8 ± 14.5
100.0 ± 0.0
94.4 ± 6.1
77.7 ± 7.0
Gain
+ 16.8
+ 0.0
+ 0.4
0.0
+ 7.2
+ 4.9
SigLIP
Baseline
69.6 ± 9.2
52.0 ± 12.3
59.2 ± 15.6
100.0 ± 0.0
84.0 ± 9.4
73.0 ± 7.6
SPARK
74.6 ± 10.3
60.1 ± 9.5
58.5 ± 12.7
100.0 ± 0.0
91.5 ± 5.8
76.9 ± 6.8
Gain
+ 5.0
+ 8.1
-0.7
0.0
+ 7.5
+ 3.9
Appendix
Table B: Experimental results on vision-based robot policy learning on MetaWorld. We report the performance of imitation learning agents on MetaWorld [ 40 ] , which are trained upon representations of CLIP [ 32 ] , DINOv2 [ 30 ] , SigLIP [ 42 ] , and Theia [ 34 ] post-trained on Kinetics-400 [ 21 ] dataset. The success rates (%) are reported for all the tasks. We report the gains of our method over the baseline.
Method
Knob1 on
Light on
Sdoor open
Ldoor open
Micro open
Average
CLIP
Baseline
23.0 ± 2.6
29.5 ± 4.7
69.5 ± 4.4
13.5 ± 3.4
22.0 ± 2.8
31.5 ± 2.4
SPARK
31.5 ± 1.9
38.5 ± 6.0
74.5 ± 11.4
18.0 ± 2.0
29.0 ± 6.2
39.8 ± 5.6
Gain
+ 8.5
+ 9.0
+ 5.0
+ 4.5
+ 7.0
+ 8.3
SigLIP
Baseline
17.0 ± 1.2
38.5 ± 1.9
75.5 ± 3.4
8.5 ± 1.0
16.5 ± 1.9
31.2 ± 0.6
SPARK
27.5 ± 5.0
43.0 ± 5.0
87.5 ± 1.9
20.0 ± 2.8
33.0 ± 5.3
42.2 ± 2.4
Gain
+ 10.5
+ 4.5
+ 12.0
+ 11.5
+ 16.5
+ 11.0
Appendix
Table C: Experimental results on vision-based robot policy learning on Franka Kitchen. We report the performance of imitation learning agents on Franka Kitchen [ 13 ] , which are trained upon representations of CLIP [ 32 ] , DINOv2 [ 30 ] , SigLIP [ 42 ] , and Theia [ 34 ] post-trained on Kinetics-400 [ 21 ] dataset. The success rates (%) are reported for all the tasks. We report the gains of our method over the baselines.
Method
Knob1 on
Light on
Sdoor open
Ldoor open
Micro open
Average
SigLIP
17.0 ± 1.2
38.5 ± 1.9
75.5 ± 3.4
8.5 ± 1.0
16.5 ± 1.9
31.2 ± 0.6
STP
14.0 ± 3.7
34.0 ± 4.3
56.5 ± 6.0
14.5 ± 2.5
12.0 ± 1.6
26.2 ± 2.1
Lossless Adaptation
17.5 ± 5.3
37.5 ± 5.3
78.5 ± 6.6
9.0 ± 2.0
12.5 ± 3.4
31.0 ± 1.6
SPARK
27.5 ± 5.0
43.0 ± 5.0
87.5 ± 1.9
20.0 ± 2.8
33.0 ± 5.3
42.2 ± 2.4
Appendix
Table D: Comparison with representation adaptation methods. All methods are initialized from SigLIP [ 42 ] . The success rates (%) are reported for all the tasks on Franka Kitchen [ 13 ] .
Visual foundation models provide strong perceptual features for robotics, but their dense representations lack explicit object-level structure, limiting robustness and controllability in manipulation tasks. We propose STORM (Slot-based Task-aware Object-centric Representation for robotic Manipulation), a lightweight object-centric adaptation module that augments frozen visual foundation models with a small set of task-aware slots for robotic manipulation. Rather than fully tuning large backbones on the task, STORM employs an efficient two-stage training strategy: few layers of object-centric representation are first trained on top of the frozen backbone through visual--semantic pretraining using language embeddings, then jointly adapted with a downstream manipulation policy for task alignement. This staged learning prevents degenerate slot formation and preserves semantic consistency while aligning perception with task objectives. Experiments on object discovery benchmarks and robotic manipulation tasks show that STORM improves control performance and generalization to visual shifts (distractors, textures, lighting) compared to directly using frozen or fine-tuned foundation model features, or existing object-centric representations. STORM serves not only as an efficient mechanism for refining generic foundation model features, but also as a novel way of injecting beneficial structural and semantic bias into policy learning.
The prevailing approach to improving robot foundation models (RFMs) relies on larger models, more robot demonstrations, and costly training at scale. We show that there exists an effective and efficient complementary approach: the right reasoning recipe can substantially improve the zero-shot task performance of existing state-of-the-art RFMs. We refer to this recipe as ARC. It consists of three key ingredients: a reasoning trace, a scalable automatic labeling pipeline, and a strategy for adapting pretrained RFMs to use these traces for control. First, we find that effective reasoning traces should be grounded in the robot's next action and explain its causal structure: why the action is appropriate and what effect it should produce. Second, we show that these traces can be generated automatically from existing demonstrations, enabling us to construct ARC-Trace-DROID from DROID without collecting new robot data. Third, we show how state-of-the-art VLAs such as π0.5 and WAMs such as Cosmos3-Nano-Policy can learn to use these traces for control, with fine-tuning and inference tailored to each model's architecture and capabilities. Using ARC, we obtain gains in zero-shot RFM performance that, to our knowledge, are unprecedented without additional robot demonstrations or foundation-scale training. The adapted models establish a new state of the art on RoboLab-120 and MolmoSpaces, with gains of up to 50 percentage points on RoboLab-Reasoning-50. On real robots, ARC improves π0.5's task success by 82.2 percentage points. Project website: https://arc-robot-reasoning.github.io/
Gokul Puthumanaillam, Tao Sun, Elie Aljalbout +5
University of Illinois Urbana-Champaign · NVIDIA · Stanford University +2
Robotic perception in unstructured environments remains challenging despite the zero-shot capabilities of foundation models such as SAM. This work attributes performance degradation to non-uniform representation shifts across transformer layers: shallow layers exhibit substantial domain gaps (CKA < 0.5), whereas deep layers transfer effectively (CKA > 0.7). Based on this observation, we propose RepSAM, a representation-guided parameter-efficient fine-tuning (PEFT) framework for adapting foundation models to robotic vision. RepSAM employs a theoretically grounded CKA-guided rank allocation strategy combined with a multi-modal fusion module for robust handling of challenging robotic scenarios, including transparent objects and cluttered scenes. Experimental evaluation across six benchmarks and robotic manipulation tasks demonstrates that RepSAM achieves 97.9% of full fine-tuning performance (89.0% vs. 90.9% mIoU) while reducing trainable parameters by 158x (from 632M to 4.0M). RepSAM outperforms DoRA by 7.9% mIoU with just 4 hours of training on a single A100 GPU (a 96x reduction from full fine-tuning, which takes 384 GPU-hours). These improvements are statistically significant (p < 0.01) and translate to a 12.0% absolute improvement in robotic manipulation success rates over the LoRA (RGB) baseline.
Wenhui Chu
Department of Computer Science and Engineering, Texas A&M University, College Station, TX, USA