Abstract
Current language-guided 3D object placement typically predicts a rigid 6-DoF pose, ignoring the physical reality that indoor placements are dictated by contact. Consequently, predicted poses often float, interpenetrate supports, or fail to deform soft surfaces. We propose Deformable Contact-Aware Placement (DCAP), a framework that reframes placement as a drop-to-equilibrium process. Rather than predicting a static pose, DCAP constructs a semantic release state that specifies the drop location, object scale, and deformable components, allowing a coupled rigid-deformable simulator to settle the object under gravity and friction. This explicitly separates semantic intent from physical realization. Evaluated on a benchmark of 2,063 prompt-asset-scene cases, DCAP significantly outperforms static baselines. On standard placements, it improves VLM Physics & Naturalness by 21.2 points and human preference by 26.5 points over the strongest raw rigid baseline, while reducing mean penetration depth by 72.3%. Compared to simulated rollout baselines, DCAP reduces residual speed by 72.1%. Ultimately, our results prove that physically accurate 3D placement requires modeling semantic intent and physical realization as strictly separate processes.
More in this topic3D scene layout generation arranges objects and structures into coherent, controllable scenes from text, images, plans, or functional goals.
See all 84 papers → Aug 17, 2026
We introduce ReRef-3D, a benchmark for language-guided single-object placement in controlled 3D tabletop scenes. It contains 33,826 instructions across 998 CLEVR-derived scenes, spanning 16 placement types and direct, one-hop, and two-hop references. Given that an instruction defines a region of acceptable placements rather than one coordinate, predictions are evaluated by relation satisfaction and physical validity. Each instruction also includes a verified natural rephrasing. We evaluate LLaVA-3D, 3D-LLM, 3D-LLaVA, LL3DA, and PlaceIt3D, with PlaceIt3D evaluated under both point-based and region-based supervision. Region-supervised PlaceIt3D achieves the highest placement accuracy at about 80% and LLaVA-3D reaches about 70%. Performance varies by relation type, but nearest and aligned remain the most difficult. Rephrasing has little effect on performance overall.
Mary Lynn Martin, Yifei Zhang, Martha Palmer +1
University of Colorado Boulder
Oct 20, 2026
Reasoning about fine-grained placement constraints, such as support and containment, is essential for realistic 3D scene creation. It remains a significant challenge for existing methods that rely on simplified object representations like oriented bounding box (OBB), which are blind to the detailed geometry. To address this limitation, we introduce GALA, a geometry-aware language model with explicit point cloud perception for controllable object placement. Our model performs autoregressively by predicting the 6D pose of one target object at a time conditioned on the scene context and placement instruction. We utilize a two-stage training strategy: the first stage aligns geometric and textual modalities through pretraining tasks, and the second stage fine-tunes the model for placement. To facilitate this task, we also curate a multi-source scene dataset covering diverse room types and fine-grained furniture arrangements, which will be open-sourced for community benefit. Experiments on our benchmark show our model significantly outperforms baselines without explicit geometric reasoning. We further develop a multi-stage plan-place-verify agent pipeline that uses the model for text-driven scene generation. Project Page: https://xccelephant.github.io/paper/gala/
Chucheng Xiang, Runze Wang, Zhi Deng +10
University of Science and Technology of China, Hefei, China · Tencent, Shenzhen, China · Tsinghua University, Shenzhen, China +1
Aug 21, 2026
Object placement is critical in image composition, requiring spatially and semantically coherent positioning of objects within diverse scenes. Existing approaches typically rely on hand-crafted rules or supervised learning on limited datasets, which restricts their generalization and interpretability, especially in open-world scenarios involving novel objects and scenes. In this work, we reformulate open-world object placement as a heuristic search task guided by reasoning from a Multimodal Large Language Model (MLLM). We introduce presto, a zero-shot, training-free framework that operates within an imaginary action space to iteratively refine object position and scale. Our coarse-to-fine search strategy ensures fast convergence, and we evaluate two decision-making variants: Metric-guided Selection and MLLM-as-a-judge. Experiments across multiple benchmarks show that presto~achieves state-of-the-art performance, particularly in previously unseen, open-world settings. Human studies further reveal that the MLLM-as-a-judge variant produces more perceptually coherent placements than metric-driven approaches, highlighting a gap between standard evaluation metrics and human visual judgment.
Weixuan Ding, Shang Liu, Hanyu Pei +1
Wuhan University, Wuhan Hubei Province, China · University of Louisville, Louisville KY, USA