Robotic manipulation in cluttered environments presents a critical challenge for automation. Recent large-scale, end-to-end models demonstrate impressive capabilities but often lack the data efficiency and modularity required for retrieving objects in dense clutter. In this work, we argue for a paradigm of specialized, decoupled systems and present Unveiler, a framework that explicitly separates high-level spatial reasoning from low-level action execution. Unveiler's core is a lightweight, transformer-based Spatial Relationship Encoder (SRE) that sequentially identifies the most critical obstacle for removal. This discrete decision is then passed to a rotation-invariant Action Decoder for execution. We demonstrate that this decoupled architecture is not only more computationally efficient in terms of parameter count and inference time, but also significantly outperforms both classic end-to-end policies and modern, large-model-based baselines in retrieving targets from dense clutter. The SRE is trained in two stages: imitation learning from heuristic demonstrations provides sample-efficient initialization, after which PPO fine-tuning enables the policy to discover removal strategies that surpass the heuristic in dense clutter. Our results, achieving up to 97.6% success in partially occluded and 90.0% in fully occluded scenarios in simulation, make a case for the power of specialized, object-centric reasoning in complex manipulation tasks. Additionally, we demonstrate that the SRE's spatial reasoning transfers zero-shot to real scenes, and validate the full system on a physical robot requiring only geometric workspace calibration; no learned components are retrained.
Figures & tables
Fig. 1: Unveiling occluded targets through strategic obstacle removal. Given a cluttered scene with objects of similar shapes and colors, our approach identifies which obstacles must be removed and in what order to access the target object (blue object), solving the spatial reasoning challenge before action execution.
Fig. 2: Unveiler system architecture. Independent training of the SRE and Action Decoder enables each component to specialize in its respective task while sharing a discrete object index at inference. The SRE processes scene and object-centric visual inputs through a transformer to select the optimal obstacle, and the Action Decoder generates rotation-invariant push-grasp parameters for the selected object.
Method
Params. (M)
Est. Inference (ms)
Efficiency ( ↑ )
Unveiler (Ours)
83.03
∼260
1.0x
VILG
156.26
∼29,300\textsuperscripta
0.093–0.0047x
GPT-4o ∗
-
>6000\textsuperscriptb
<0.043 x
ThinkGrasp +
-
>6000\textsuperscriptb
<0.043 x
TABLE I: Computational Efficiency Comparison
Fig. 3: Example scenes illustrating occlusion levels of the target object. (Left) The target object is partially occluded, with some visible surface area. (Right) The target object is fully occluded, entirely covered by surrounding objects, and not directly visible in the input view.
Clut.
Occl.
GPT-4o
CLIP-Ground.
VILG
ThinkGrasp
ACT
PPG
Heur
Unveiler (Ours)
%
Steps
%
Steps
%
Steps
%
Steps
%
Steps
%
Steps
%
Steps
%
Steps
2–6
Partial
86.0
1.50
80.5
1.67
75.0
3.0
73.3
3.00
45.2
1.14
40.6
4.42
87.3
2.35
96.1
1.32
2–6
Full
66.7
2.50
60.3
2.11
50.0
2.86
53.3
2.25
40.0
1.00
50.3
4.27
67.5
2.30
89.3
1.87
6–9
Partial
80.0
1.81
67.2
3.10
80.0
3.17
66.7
3.29
40.0
1.00
7.5
6.00
67.5
2.55
97.6
1.43
6–9
Full
60.0
4.22
66.7
4.20
40.0
3.33
60.0
3.78
10.0
1.00
4.4
3.00
47.8
3.43
90.0
3.31
9–12
Partial
66.7
3.10
53.0
4.38
46.7
2.86
46.7
3.42
25.7
1.00
4.4
4.00
50.0
3.40
92.6
2.86
TABLE II: Performance comparison of GPT-4o, CLIP-Grounding, VILG, ThinkGrasp, ACT, PPG, Heur, and Unveiler (ours) across different clutter densities and occlusion levels. Metrics: Task Completion (%) and Average Planning Steps.
Fig. 4: Scene overlay with selection probabilities. Target outlined in red, selected obstacle in green.
Method
Density
Partial
Full
%
Steps
%
Steps
No SRE
6–9
96.0
2.10
76.0
3.39
Multi-Obstacle SRE
6–9
73.3
3.44
60.0
4.90
Binary Mask Input
6–9
56.7
2.94
36.7
4.55
Unveiler (Ours)
6–9
97.6
1.43
90.0
3.31
No SRE
9–12
84.0
2.91
46.0
6.39
TABLE III: Ablation study. Architectural variants evaluated at 6–9 object density; density generalization shown for No SRE vs. full Unveiler. Metrics: Task Completion (%) and Average Planning Steps.
Method
2 - 4 (%)
5 - 8 (%)
CLIP-Grounding
65
37
GPT-4o
40
26
SRE (Ours)
60
54
TABLE IV: Object Selection Accuracy on Real Scenes ( N=35 )
Retrieving objects buried beneath clutter is both challenging and time-consuming, as complex support relationships make manipulation particularly difficult. Existing methods either focus on support relations and rely on sequential grasping to remove occluding objects, or perform preparatory actions such as pushing to facilitate subsequent grasps. However, these approaches are often inefficient and treat physical interactions as isolated auxiliary steps. In this paper, we propose RetrDex, an efficient framework for dexterous arm-hand systems to learn object retrieval in cluttered scenes. Our approach leverages large-scale parallel reinforcement learning (RL) in diverse cluttered scenes and incorporates a spatially aware representation that encodes occlusion patterns and spatial relationships among the target, the dexterous hand, and surrounding clutter. This representation enables the policy to develop diverse manipulation skills (e.g., pushing, stirring, and poking) that actively clear occluders. We evaluate RetrDex on 16 household objects across varied clutter configurations, and obtain strong retrieval performance and efficiency on both seen and unseen targets. Furthermore, we demonstrate successful zero-shot transfer to a real-world dexterous multi-fingered robot system, validating the practical applicability of our method. Videos can be found on our project website: https://RetrDex.github.io.
Fengshuo Bai, Yu Li, Jie Chu +5
Shanghai Jiao Tong University · PKU-PsiBot Joint Lab · Peking University +2
Robots that follow natural-language instructions unlock a wide range of autonomous applications in households, warehouses, and labs. Real-world environments are diverse and often heavily cluttered, requiring robots to plan over multiple steps, act, and replan as they work toward a goal. We present CLARO (CLutter-Aware Robot Orchestrator), a VLM agent that orchestrates a set of perception and manipulation skills: open-vocabulary object detection, segmentation, 6-DoF grasping, and non-prehensile pushing to locate and retrieve a language-specified target in clutter. CLARO maintains a memory of its past actions to support long-horizon execution, and emits manipulation targets (grasp poses/3D waypoints) allowing transfer across robot arms without task-specific training. We evaluate CLARO against a range of methods on a dynamic simulation environment and standard image-based benchmarks, demonstrating strong generalization across object classes, lighting, and clutter levels. CLARO attains an average success rate of 89.1%, a relative improvement of over 16% on the strongest competing method in high cluttered scenes. We further analyze performance along grounding and grasping axes to characterize where each method succeeds and fails. Finally, we deploy CLARO on a robot arm across diverse cluttered scenes, achieving a success rate of 88.8%, and demonstrating strong sim2real generalizability.
Sungsu Kim, Vineet Bhat, Prashanth Krishnamurthy +2
Vision-Language-Action (VLA) models demonstrate impressive zero-shot generalization but frequently suffer from a "Precision-Reasoning Gap" in cluttered environments. This failure is driven by background-induced feature dilution, where high-frequency semantic noise corrupts the geometric grounding required for precise manipulation. To bridge this gap, we propose Concept-Gated Visual Distillation (CGVD), a training-free, model-agnostic inference framework that stabilizes VLA policies. CGVD operates by parsing instructions into safe and distractor sets, utilizing a two-layer target refinement process--combining cross-validation and spatial disambiguation--to explicitly penalize false positives and isolate genuine manipulation targets. We then process the scene via Fourier-based inpainting, generating a clean observation that actively suppresses semantic distractors while preserving critical spatial geometry and visual proprioception. Extensive evaluations in highly cluttered manipulation tasks demonstrate that CGVD prevents performance collapse. In environments with dense semantic distractors, our method significantly outperforms state-of-the-art baselines, achieving a 77.5% success rate compared to the baseline's 43.0%. By enforcing strict attribute adherence, CGVD establishes inference-time visual distillation as a critical prerequisite for robust robotic manipulation in the clutter.
Sangmim Song, Sarath Kodagoda, Marc Carmichael +1
University of Technology Sydney, NSW, Australia · Western Sydney University, NSW, Australia