Vision-Language-Action (VLA) models demonstrate impressive zero-shot generalization but frequently suffer from a "Precision-Reasoning Gap" in cluttered environments. This failure is driven by background-induced feature dilution, where high-frequency semantic noise corrupts the geometric grounding required for precise manipulation. To bridge this gap, we propose Concept-Gated Visual Distillation (CGVD), a training-free, model-agnostic inference framework that stabilizes VLA policies. CGVD operates by parsing instructions into safe and distractor sets, utilizing a two-layer target refinement process--combining cross-validation and spatial disambiguation--to explicitly penalize false positives and isolate genuine manipulation targets. We then process the scene via Fourier-based inpainting, generating a clean observation that actively suppresses semantic distractors while preserving critical spatial geometry and visual proprioception. Extensive evaluations in highly cluttered manipulation tasks demonstrate that CGVD prevents performance collapse. In environments with dense semantic distractors, our method significantly outperforms state-of-the-art baselines, achieving a 77.5% success rate compared to the baseline's 43.0%. By enforcing strict attribute adherence, CGVD establishes inference-time visual distillation as a critical prerequisite for robust robotic manipulation in the clutter.
Figures & tables
Fig. 1: Comparison of manipulation task execution in cluttered environments. While a standard VLA model (left) struggles with object confusion in a highly cluttered scene, our CGVD approach (right) successfully identifies and places the target object (”spoon”) on the towel.
Fig. 2: Overview of the CGVD pipeline. Stage 1: The language instruction is parsed to extract a safe set tar and a distractor set. Stage 2: SAM3 segments both sets independently, producing a safe-set mask and a distractor mask via dual-channel segmentation. Stage 3: Set-theoretic gating subtracts the safe-set mask from the distractor mask, and LaMa inpaints the resulting regions to produce a distilled observation passed to the VLA policy.
Fig. 3: Success rate vs. number of distractors. Left: semantic distractors. Right: random distractors. Top: spoon on towel. Bottom: carrot on plate. Dashed lines represent the baseline VLA; solid lines represent +CGVD. Colors denote specific model architectures. To ensure statistical significance, each data point represents the average success rate over 200 independent evaluation rollouts (20 episodes × 10 random seeds), totaling 19,200 episodes for the results visualized in this figure.
Fig. 4: Qualitative Analysis of Attention Repair. (Top) The baseline policy suffers from attention dispersion, focusing on the distractors rather than the spoon. (Bottom) Our CGVD method inpaints the distractors, forcing the attention mechanism to collapse onto the true target.
Simple Prompt
Complex Prompt
# Distractors
π0
CGVD
Δ
π0
CGVD
Δ
0
86.0
90.0
+4.0
85.0
87.0
+2.0
1
80.0
78.0
− 2.0
74.0
69.0
− 5.0
2
73.0
87.0
+14.0
69.0
77.0
+8.0
3
68.0
75.0
+7.0
64.0
74.0
+10.0
4
75.0
87.0
+12.0
57.0
73.0
+16.0
TABLE I: Attribute Distractor Sensitivity. Success rates on Put spoon on towel with attribute distractors. Results are averaged over 10 random seeds (20 episodes per seed). CGVD shows stronger average robustness on both simple and complex prompts, though both methods exhibit non-monotonic variance at low distractor counts.
Configuration
SR (%)
Baseline (no CGVD)
43.0
CGVD (full pipeline)
77.5
– Mean-color fill
56.5
– Two-layer target refinement
65.0
– Robot mask protection
73.0
TABLE II: Ablation study on the Spoon on Towel task ( π0 , 18 semantic distractors). Each row removes one component from the full CGVD pipeline. To ensure statistical significance, every configuration was evaluated over 200 episodes (20 episodes across 10 seeds).
Phase
Base π0 (ms)
CGVD (ms)
Initialization ( t=0 )
—
4,914
Execution ( t>0 )
317
421
TABLE III: System Latency. CGVD concentrates segmentation and inpainting at t=0 ; runtime compositing adds moderate overhead at the VLA control frequency.