cs.RONov 3, 2026

VARP: A Visual Adaptive Rearrangement Planner for Non-Monotone Object Rearrangement Planning

Authors: Hanwen Ren, Meng Zheng, Zhongpai Gao, Anwesa Choudhuri, Benjamin Planche, Terrence Chen, Ziyan Wu

Abstract

Object rearrangement planning in cluttered environments is often non-monotone, requiring objects to be temporarily relocated before reaching their final destinations. Although search-based planners can solve such tasks, they incur substantial computational cost and require expensive replanning when the environment changes. We propose the Visual Adaptive Rearrangement Planner (VARP), which learns non-monotone rearrangement directly from visual state representations generated from expert demonstrations. VARP predicts pick-and-place actions from the current and goal visual state representations, and incorporates an Action Refinement Strategy (ARS) to improve action feasibility and a Progress Enforcement Strategy (PES) to mitigate progress stagnation. By continuously updating actions from the latest visual state representations, VARP adapts online to disturbances without replanning from scratch or performing online combinatorial search. In challenging rearrangement benchmarks, VARP achieves a 98.3% success rate with only 0.4 s planning time per task, 97.5% success under execution-time disturbances, and 94.5% under continuous severe disturbances, while generalizing effectively to unseen object counts. These results demonstrate that VARP provides a scalable and robust solution for non-monotone object rearrangement.

Figures & tables

Explore similar work

Aug 27, 2026

Embodied Scene Rearrangement Planning

This paper introduces Embodied Scene Rearrangement Planning (ESRP), a novel task requiring embodied agents to rearrange furniture in 3D scenes to match a target configuration using only egocentric observations and a top-down target layout. Unlike prior rearrangement tasks, ESRP precludes global state access and introduces mutual object occlusions, reflecting the practical constraints of real-world robotic deployment. These factors make aligning partial egocentric observations with the global target layout particularly challenging for long-horizon planning. To facilitate research, we present ESRP-Bench, a comprehensive benchmark built on OmniGibson featuring over 5,400 scene pairs and 8,200 objects. We define three multi-level metrics to evaluate rearrangement quality and provide four baselines: a hierarchical task-and-motion planning method, a vision-language-model-based method, and two learning-based approaches (IL and RL). Experimental results demonstrate that current methods struggle to complete the task efficiently, highlighting ESRP as a challenging frontier for embodied agents in scene understanding and long-horizon task planning. This work serves as a stepping stone toward deploying intelligent agents in real-world scenarios. Project page: https://pie-lab.cn/ESRP/.
Nov 12, 2026

Visibility-Aware Task and Motion Planning with Task-Centric Perception

Long-horizon manipulation under uncertainty remains a challenging problem in robotics. We present \ourmethod{}, an algorithm for Visibility-Aware Task and Motion Planning. \ourmethod{} extends existing deterministic task and motion planning solvers to partially observable object rearrangement tasks, while still being able to take advantage of the efficiency and compositional generalization over long horizons of the base solver. This extension interleaves sensing actions into a plan skeleton that assumed full observability, and uses a novel persistent object-based memory to fuse observations into object mesh hypotheses and an observed occupancy map. We evaluate \ourmethod{} on real-world tabletop rearrangement tasks which feature heavy occlusion and partial visibility and demonstrate that it outperforms baselines that do not perform active perception or have persistent memory.
Nov 9, 2026

Learning Contextual Motion Constraints for Object Rearrangement Exploration

The motion of an articulated part in cluttered surroundings is constrained not only by an object's intrinsic joint limits but also by extrinsic obstacles. However, existing approaches primarily consider object articulation properties alone. Here we present Motion Constraint Prediction Network (MoCo-Net), a two-stream neural network that jointly predicts the continuous motion ranges of articulated parts and obstacle heatmaps highlighting potential collision regions from partial-view point cloud observation. By capturing both internal and external motion constraints, MoCo-Net produces more accurate feasible motion range prediction and obstacles recognition, improves generalization for novel objects in new scenes, and enables robots to construct a scene configuration space for efficient exploration of rearrangement actions. Our experiments show that MoCo-Net's two-stream design enhances each other to deliver more accurate predictions, which subsequently form a compact configuration space for a customized planner to efficiently rearrange the scene.