VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation

Xuanyou Chen; Jinyu Chen; Glen Chou

cs.RONov 2, 2026

VerifLM: Physically Verified Image-Edit-Guided Goal Pose Generation for Robotic Manipulation

Authors: Xuanyou Chen, Jinyu Chen, Glen Chou

Organizations: The authors are with the Georgia Institute of Technology, Atlanta, GA 30308, USA.

Abstract

Many manipulation goals are easier to visualize than to specify numerically through explicit geometric constraints. We present VerifLM, a framework that converts image-edited goal configurations into physically verified 6-DoF object poses for planning. Using an edited image, VerifLM recovers an initial relative pose between an active object and a passive object, then prompts a vision-language model to generate axis-wise sampling constraints and a task-specific validation program. Candidate poses are evaluated in a physics simulator, and the VLM selects the most task-consistent retained candidate. To operate without predefined object meshes or physical parameters, VerifLM reconstructs metric object geometry from RGB-D observations and constructs multiple hypothetical simulations with varying physics and geometry. Parallel-world validation evaluates candidates across geometry--physics variations and ranks them by the number of worlds in which they satisfy the validation program. We evaluate VerifLM on nine manipulation tasks. VerifLM achieves the highest task success on eight of nine tasks, with the largest performance gain on goals requiring precise orientation and contact relationships.

Figures & tables

Fig. 1: Illustration of ReKep and OmniManip constraints on representative tasks. Both methods perform well when object orientation is weakly constrained, as in Mug on Plate, where yaw does not affect success. However, they struggle when the intended goal pose requires a highly specific orientation or contact arrangement. In Plate on Stand , OmniManip recovers the desired orientation but not the position, causing penetration.

Fig. 2: Common hallucination patterns in VLM-edited goal images. Given the same input scene, image editing may alter the camera viewpoint, distort object geometry, displace the passive object, or generate visually plausible but physically invalid contact.

Fig. 3: Overview of VerifLM, which converts an edited goal image into a physically verified object goal pose and executes a trajectory to realize it.

Criterion	Initial pose	Every settling step	Final state	geometric_only	Source
Initial penetration	Yes	No	No	No	Configured
Final penetration	No	No	Yes	Yes	Configured
Forbidden contacts	Yes	Yes, immediate rejection	Yes	Yes	VLM
Required contacts	No	No	Yes	Yes	VLM
Position drift	No	No	Yes	No	VLM
Orientation drift	No	No	Yes	No	VLM

TABLE I: Validation criteria, evaluation timing, and whether manually configured or VLM-provided.

Single-Stage Task	Success criterion
Mug on Rack	Mug hangs from the rack by its handle.
Mug on Plate	Mug is upright near the plate center.
Mug on Stand	Mug is placed on the plate stand inverted, covering at least two notches.
Plate on Stand	Plate is securely supported in a pair of opposing slots, either directly corresponding or offset by one slot.
Pour Tea	Teapot reaches a valid pouring pose with its spout over the mug.
Lid on Teapot	Lid is correctly seated on the teapot.

TABLE II: Manipulation benchmarks and task-level success criteria.

Single-stage
Task	ReKep [ 5 ]	OmniManip [ 6 ]	Goal-VLA [ 11 ]	Ours (with predefined meshes)		Ours (mesh-free-input)
	Task Planning	Task Planning	Task Planning	Task Planning	Execution	Task Planning	Execution
Mug on Rack	4/24 (16.67% $\pm$ 11.79%)	6/24 (25.00% $\pm$ 10.21%)	11/24 (45.83% $\pm$ 5.89%)	16/24 (66.67% $\pm$ 15.59%)	16/24 (66.67% $\pm$ 15.59%)	17/24 (70.83% $\pm$ 11.79%)	14/24 (58.33% $\pm$ 11.79%)
Mug on Plate	16/24 (66.67% $\pm$ 15.59%)	19/24 (79.17% $\pm$ 5.89%)	20/24 (83.33% $\pm$ 11.79%)	22/24 (91.67% $\pm$ 11.79%)	22/24 (91.67% $\pm$ 11.79%)	20/24 (83.33% $\pm$ 15.59%)	20/24 (83.33% $\pm$ 15.59%)
Mug on Stand	10/24 (41.67% $\pm$ 11.79%)	7/24 (29.17% $\pm$ 11.79%)	15/24 (62.50% $\pm$ 17.68%)	20/24 (83.33% $\pm$ 15.59%)	20/24 (83.33% $\pm$ 15.59%)	18/24 (75.00% $\pm$ 17.68%)	17/24 (70.83% $\pm$ 15.59%)
Plate on Stand	0/24 (0.00% $\pm$ 0.00%)	6/24 (25.00% $\pm$ 10.21%)	10/24 (41.67% $\pm$ 5.89%)	17/24 (70.83% $\pm$ 11.79%)	16/24 (66.67% $\pm$ 15.59%)	16/24 (66.67% $\pm$ 11.79%)	13/24 (54.17% $\pm$ 11.79%)

TABLE III: Task-planning and execution success across nine benchmarks.

Task	Variant	Success Rate	$\mathbf{N_{\mathrm{consistent}}}$
			$N_{s}=250$	$N_{s}=500$	$N_{s}=1000$	$N_{s}=2000$
Mug on Rack	Full method ( $\mathcal{S}+\mathcal{V}$ )	7/12	3.17	4.50	4.92	5.50
	Default $\mathcal{V}$	6/12	0.92	2.08	2.67	3.08
	Default $\mathcal{S}$	4/12	0.42	0.75	1.50	2.17
	Default $\mathcal{S}+\mathcal{V}$	2/12	0.33	0.42	0.67	0.75
Plate on Stand	Full method ( $\mathcal{S}+\mathcal{V}$ )	8/12	5.67	7.17	7.92	7.92

TABLE IV: Ablation of sampling constraints

\mathcal{S}

+ validation program

\mathcal{V}

Panel of Figure 8 — Fig. 4: Task-consistent and task-inconsistent candidates.

Fig. 5: Example showing sources of error that could arise when extracting a goal pose from the VLM-edited image. First, the image editor elongates the rack’s branch and produces a physically infeasible contact arrangement. Second, the perception module with VGGT and FoundationPose overestimates the distance between the mug and the rack.

Task	Image-Derived Pose Only	Full Method
Mug on Rack	10/24	16/24
Mug on Plate	19/24	22/24
Mug on Stand	11/24	20/24
Plate on Stand	5/24	17/24
Pour Tea	14/24	16/24
Lid on Teapot	16/24	18/24

TABLE V: Ablation of the pose-refinement process.

Task	Single-World Validation	Parallel-World Validation
Mug on Rack	11/24	14/24
Mug on Plate	19/24	20/24
Mug on Stand	16/24	17/24
Plate on Stand	9/24	13/24
Pour Tea	15/24	16/24
Lid on Teapot	7/24	14/24

TABLE VI: Ablation of parallel-world validation.

Robotic Manipulation

Robotic manipulation studies how robots perceive, plan, and control physical interactions to grasp, move, assemble, and use objects.

See all 1,765 papers

Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation
Oct 8, 2026Lonnie Tankersley, Noorbakhsh Amiri Golilarz, Soroush KorivandRobotic Manipulation Robot Task Planning
dept. of Aerospace Engineering, Mississippi State University, Starkville, MS · dept. of Computer Science, The University of Alabama, Tuscaloosa, AL · dept. of Mechanical Engineering, Mississippi State University, Starkville, MS
Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents
Apr 10, 2026Sangwon Baik, Gunhee Kim, Mingi Choi +1Language-Conditioned Robot Manipulation Object Pose Estimation
Seoul National University · RLWRLD
AntiGrounding: Executable Robot Trajectories as Visual Prompts for VLM-Guided Manipulation
Jun 14, 2025Wenbo Li, Yiteng Chen, Wenhao Li +1Trajectory Generation Language-Conditioned Robot Manipulation
School of Software Engineering, South China University of Technology

Oct 8, 2026J/K move · Enter open · S save

Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation

Lonnie Tankersley, Noorbakhsh Amiri Golilarz, Soroush Korivand

dept. of Aerospace Engineering, Mississippi State University, Starkville, MS · dept. of Computer Science, The University of Alabama, Tuscaloosa, AL · dept. of Mechanical Engineering, Mississippi State University, Starkville, MS

Robotic Manipulation Robot Task Planning Large Language Model-Based Robot Planning Zero-Shot Robotic Manipulation VLMs for Robotics

Open paper AskPDF arXiv

Large language models can translate natural-language instructions into plausible manipulation plans, but robotic execution requires physically grounded decisions that respect robot embodiment, workspace geometry, reachability, collision constraints, and grasp feasibility. This paper presents a hardware-aware large language model-vision-language model (LLM-VLM) manipulation framework for a simulated Franka Research 3 robot in NVIDIA Isaac Sim. The framework separates symbolic reasoning from metric execution: an API-hosted LLM planner receives a structured scene summary and generates only symbolic task information, including the object to manipulate, target relation, approach preference, and objects to avoid. A local Qwen3-VL 8B VLM verifier supports semantic confirmation of ambiguous objects and open-container surrogates. Both the LLM planner and the VLM verifier are prohibited from generating robot coordinates, grasp poses, keepout dimensions, gripper commands, trajectory waypoints, or success labels. Instead, these decisions are handled by deterministic grounding and execution modules that use the robot model, scene geometry, MoveIt 2 collision checking, inverse kinematics, grasp validation, release verification, and final visual assessment. Execution feedback is returned only as symbolic context for replanning. The framework is evaluated through simulated digital-twin trials involving pickup, occluded-source access, keepout reasoning, relation placement, and sequential manipulation tasks.

Abstract

Robotic Manipulation

Explore similar work

Embodied Zero-Shot Task Planning via Hardware-Aware LLM-VLM Integration for Grounded Robotic Manipulation

Text-Guided 6D Object Pose Rearrangement via Closed-Loop VLM Agents

AntiGrounding: Executable Robot Trajectories as Visual Prompts for VLM-Guided Manipulation