cs.CVSep 7, 2026
SaveGIFT: Goal-Injected Fine-Tuning for Efficient Manipulation Policy Adaptation
Organizations: College of Artificial Intelligence, Nanjing University of Aeronautics and Astronautics, Nanjing, 211106, China · The Key Laboratory of Brain-Machine Intelligence Technology, Ministry of Education, Nanjing, 211106, China.
Abstract
Compared with relying solely on initial observations and language instructions, predicting goal images with generative models as high-level visual guidance can significantly enhance the robustness of Vision-Language-Action (VLA) models. However, most existing foundation models have not systematically incorporated goal image conditioning due to the high computational training cost. To this end, we propose Goal-Injected Fine-Tuning (GIFT), a lightweight and efficient fine-tuning framework that seamlessly integrates generated goal images into multiple representative pretrained VLA models. Our approach introduces goal image features into observations via a zero-initialized convolution which progressively grows parameters from zero and prevents harmful noise from disrupting the pretrained policy during fine-tuning. As training proceeds, goal information is gradually incorporated, enabling efficient goal understanding without disrupting model stability. We further introduce a refined image editing method to generate semantically and visually consistent goal images from initial observations and task instructions. Experiments show that goal-aware VLA models achieve substantial performance gains across tasks: with only a single epoch of fine-tuning, GIFT outperforms the base model by 6.0% and 13.4% on two SIMPLER settings, and by 4.7% on LIBERO, demonstrating both efficiency and effectiveness.
Figures & tables
Figure 1: Overview of the proposed GIFT framework. The image editing model first generates a goal image based on the initial observation and given instruction. The generated goal is then reused throughout policy execution to guide the VLA model at each timestep.
Figure 2: Overview of our proposed Goal-Injected Fine-Tuning framework. The goal image is encoded and injected into the VLA model via zero convolution, enabling efficient goal-image condition.
Figure 3: Illustration of the proposed Image Editing for Robotic Goal pipeline. Given an observation and instruction, the model combines the inversed latent image with random noise in a T2I DiT and iteratively denoises them to generate an instruction-aligned goal image.
| Method | Visual Matching | Variant Aggregation | ||||||||
| Grasp | Move | Drawer | PutInDrawer | Avg. | Grasp | Move | Drawer | PutInDrawer | Avg. | |
| RT-1 | 85.7 | 44.2 | 73.0 | 6.5 | 52.4 | 89.8 | 50.0 | 32.2 | 2.6 | 43.7 |
| RT-1-X | 56.7 | 31.7 | 59.7 | 21.3 | 42.4 | 49.0 | 32.3 | 29.4 | 10.1 | 30.7 |
| RT-2-X | 78.7 | 77.9 | 25.0 | 3.7 | 46.3 | 82.3 | 79.2 | 35.4 | 20.6 | 54.4 |
| Octo-Base | 17.0 | 4.2 | 22.7 | 0.0 | 11.0 | 0.6 | 3.1 | 1.1 | 0.0 | 1.2 |
| OpenVLA | 18.0 | 56.3 | 63.0 | 0.0 | 34.3 | 60.8 | 67.7 | 28.8 | 0.0 | 39.3 |
Table 1: Comparison of our approach with existing VLA models on the Google robot across four tasks in two SIMPLER settings. Both settings include four tasks: (1) Pick coke can, (2) Move near, (3) Open/close drawer, and (4) Open top drawer and place apple.
| Method | Task1 | Task2 | Task3 | Task4 | Avg. |
| RT-1-X | 0.0 | 4.2 | 0.0 | 0.0 | 1.1 |
| Octo-Base | 15.8 | 12.5 | 0.0 | 41.7 | 17.5 |
| SuSIE | 34.7 | 45.5 | 6.9 | 40.2 | 31.9 |
| GHIL-Glue | 54.2 | 48.6 | 11.1 | 47.2 | 40.3 |
| OpenVLA | 4.2 | 0.0 | 0.0 | 12.5 | 4.2 |
| OpenVLA+GIFT | 9.7 | 0.0 | 0.0 | 31.7 | 10.4 |
Table 2: Evaluation results on the WidowX robot in the SIMPLER Visual Matching setting. Four tasks are: (1) Put spoon on towel (2) Put carrot on plate (3) Stack green block on yellow block and (4) Put eggplant in yellow basket. Each task is evaluated 3 times.
| Method | Spatial | Object | Goal | Long | Avg. |
| Diffusion Policy | 78.3 | 92.5 | 68.3 | 50.5 | 72.4 |
| Octo-Base | 78.9 | 85.7 | 84.6 | 51.1 | 75.1 |
| TraceVLA | 84.6 | 85.2 | 75.1 | 54.1 | 74.8 |
| OpenVLA | 84.7 | 88.4 | 79.2 | 53.7 | 76.5 |
| OpenVLA+GIFT | 89.5 | 90.6 | 88.6 | 56.0 | 81.2 |
Table 3: Evaluation results on the Franka Emika Panda robot in the LIBERO benchmark across four task suites: Spatial, Object, Goal, and Long-horizon tasks.
Figure 4: Real-world deployment setup and evaluation results. Left: Real-world experimental environment. Right: Task performance comparison between and the variant with GIFT across ID and OOD scenarios.
Figure 5: Qualitative comparison of visual editing results across various goal-directed manipulation tasks, comparing our method with existing approaches (SuSIE and Ghil-Glue). From top to bottom, the rows show: the instruction, the initial state, the edited result generated by SuSIE, Ghil-Glue, our method, and the ground-truth (GT) image.
Figure 6: Qualitative comparison among the baseline, GIFT (Ours), and CoT-VLA on the task “put the carrot on the plate,” where both GIFT and CoT-VLA are trained on the same baseline for only a single epoch.
Figure 7: Ablation study results. (a) shows the effect of Observation-structure Injection on visual editing. (b) presents quantitative comparisons on SIMPLER Visual Matching across four configurations: baseline, GIFT without Semantic Contrast, GIFT with ZeroConv replaced by FiLM, and GIFT.
| Task | Degraded | SuSIE | Turbo | Ours | GT Goal |
| Grasp | 52.1 | 66.7 | 80.0 | 87.6 | 90.3 |
| Move | 45.7 | 55.9 | 79.6 | 88.4 | 88.9 |
| Drawer | 25.5 | 29.6 | 70.9 | 71.7 | 75.3 |
| Place | 41.1 | 35.6 | 72.1 | 75.9 | 80.5 |
| Stack | 10.6 | 0.0 | 38.3 | 41.1 | 52.5 |
| Spatial | 50.4 | 48.6 | 85.6 | 89.5 | 89.9 |
Table 4: Comparison of GIFT success rates (%) under varying goal image qualities. Evaluated tasks span across the SIMPLER (first five rows) and LIBERO (last four rows) benchmarks. The GIFT injection module is fixed across all settings.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | BridgeDataset V2 | Fractal | LIBERO |
| CogACT | 24.0 | 43.2 | – |
| OpenVLA | 24.0 | 44.5 | 9.6 |
| SpatialVLA | 20.0 | 36.0 | – |
Table 5: Training time (hours) for a single epoch across different datasets with 4 NVIDIA L20 GPUs.
Figure 8: Qualitative results on the SIMPLER simulation with the WidowX robot. Each row illustrates a manipulation task, showing the initial state, the goal image, and intermediate execution frames. Our method generates semantically accurate and visually coherent goal images that align with the task instruction, effectively guiding the robot through various object-centric tasks such as precise placement, stacking, and goal-oriented repositioning.
Figure 9: Visualizations of our model’s performance across four representative tasks in the LIBERO simulation environment. Each row corresponds to a specific task instruction from one of the four LIBERO suites (Spatial, Object, Goal, and Long). For each task, we show the initial observation, the edited goal image and several intermediate execution frames. Our proposed model demonstrates the ability to produce semantically meaningful goals and successfully guide the robot through multi-step manipulation sequences.
Figure 10: Qualitative real-robot rollouts under in-domain (ID) and out-of-distribution (OOD) lighting conditions. We show two tasks, “pick up the bottle” and “put the bottle into the basket”. Each row visualizes a sequence of third-view observations during policy execution. The robot successfully completes both tasks under ID lighting and maintains robust goal-directed behavior under dim OOD lighting.
Figure 11: Qualitative results on the SIMPLER simulation environment using the Google Robot. Each row corresponds to a manipulation task evaluated under both Visual Matching (VM) and Variant Aggregation (VA) settings. For each task, we visualize the initial observation, the goal image edited and intermediate execution frames.
| Modalities | Epochs | Grasp | Stack | Stability |
| V + L | – | 80.8 | 15.0 | – |
| V + L + G | 1 | 86.9 | 43.1 | Stable |
| V + G | 1 | 0.0 | 0.0 | Unstable |
Table 6: Input-modality ablation on representative SIMPLER tasks. Removing language instructions severely degrades performance even when goal images are provided, indicating that language remains essential for stable goal-aware adaptation.
| Task | Seed 42 | Seed 22 | Seed 62 | Mean Std |
| Grasp | 87.9 | 88.3 | 87.0 | |
| Move | 86.9 | 85.7 | 86.1 | |
| Drawer | 64.7 | 62.9 | 63.0 |
Table 7: Robustness of GIFT to the stochasticity of goal-image generation under three random seeds in the SIMPLER Visual Matching setting. The performance remains highly consistent across seeds, with standard deviations below 1% on all tasks.
| Model | Params | Trainable | FLOPs | Mem | Step Time | Edit | Steps (B / G) | Ep. Cost (B / G) | Overhead |
| 3B+20M | 100M | 0.85T | 7.3G | 73ms+2ms | 4.2s | 52.4 / 50.8 | 6.45s / 10.55s | +63.7% | |
| CogACT | 7B+72M | 150M | 4.32T | 15.8G | 156ms+4ms | 4.2s | 85.3 / 77.5 | 14.16s / 17.38s | +22.7% |
| OpenVLA | 7B+72M | 200M | 4.29T | 15.5G | 75ms+4ms | 4.2s | 125.5 / 113.6 | 10.67s / 14.31s | +34.1% |
| SpatialVLA | 4B+10M | 630M | 1.94T | 8.5G | 97ms+4ms | 4.2s | 99.7 / 95.3 | 10.67s / 14.78s | +38.5% |
Table 8: System-level computational overhead of GIFT across baseline policies. Profiling is conducted on a single NVIDIA L20 GPU in bf16 precision. Params” and Step Time” report base model costs plus GIFT’s overhead. B / G” denotes Base / GIFT. Edit” is a one-time per-episode cost.
Figure 12: Qualitative results of edited goal images generated by our method ( Ours ) across diverse manipulation tasks from our test dataset. Each column represents a distinct task, with the initial state shown in the first row. Our framework successfully produces visually and semantically aligned goal states, preserving spatial and object-specific cues necessary for grounding manipulation actions, even under complex instructions and cluttered scenes.
More in this topic
Vision-Language-Action Models
Vision-language-action models turn visual observations and natural-language instructions into actions for robots and other embodied agents.
See all 1,154 papers