PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly

Kulunu Samarawickrama; Roel Pieters

cs.ROAug 20, 2026

PVRA: A Pointwise Key-point Voting Framework for Robotic Assembly

Authors: Kulunu Samarawickrama, Roel Pieters

Organizations: Tampere University, Finland

Abstract

Modern computer vision has enabled partial autonomy in robotic assembly manipulation. However, performing autonomous manipulation of a progressive assembly demands a more specific set of skills, in addition to perceiving the objects. Through a comparative analysis of research in the associated domains, we deduce that object-centric perception must advance towards learning assembly dependencies to predict meaningful actionable outputs for autonomous assembly manipulation. Subsequently, we present a 3D keypoint-based modular learning framework to learn assembly dependencies to infer actionable outputs given a RGB-D input of an assembly scene. We train and evaluate our trained network on an assembly pose estimation dataset and compare it against object-centric baselines with an augmented set of metrics for progressive assemblies.

Figures & tables

Figure 1: Proposed 3D keypoint-based learning framework for assembly cognition. MLP denotes Multi Layer Perceptrons.

Metric / Method	PVRA	ICP-GT	ICP-PVRA	FP-GT	FP-PVRA
Samples	280	215	227	282	227
SSA	0.833	–	–	–	–
Target AUC	0.745	0.576	0.521	0.941	0.770
Assembly AUC	0.790	0.416	0.420	0.921	0.881
SLA-AUC	0.757	0.379	0.337	0.927	0.727
Step [email protected]	0.893	0.344	0.286	0.989	0.771

Table 2: Quantitative evaluation results for the Nema17 test split using symmetry-aware normalized MSSD. ICP-GT denotes CAD-ICP-PCA with annotation masks, ICP-PVRA denotes CAD-ICP-PCA with masks derived from PVRA segmentation, FP-GT denotes FoundationPose with annotation masks, and FP-PVRA denotes FoundationPose with masks derived from PVRA segmentation.

Figure 2: Step-level threshold accuracy curves.

Robotic Assembly

Robotic assembly studies how robots perceive, plan, and join parts reliably, from precision insertion to collaborative construction.

See all 88 papers

AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly
Apr 10, 2026Zhi Jing, Jinbin Qiao, Ouyang Lu +53D Spatial Reasoning Robotic Assembly
Fudan University† · Institute of Artificial Intelligence (TeleAI), China Telecom† · Tianjin University +3
State-of-the-Art in Learning-by-Demonstration with Passive Observation for Industrial Assembly Automation
Sep 7, 2026David Koetter, Oliver Petrovic, Christian BrecherGeneralization in Robotic Manipulation Imitation Learning
RWTH Aachen University, Laboratory for Machine Tools and Production Engineering, Steinbachstraße 19, 52074 Aachen, Germany · Fraunhofer Intitute for Production Technology IPT, Steinbachstraße 17, 52074 Aachen, Germany
Generalizable Robotic Insertion with World Models
Sep 23, 2026Nicklas Hansen, Iretiayo Akinola, Yijie Guo +7Zero-Shot Robotic Manipulation World Models for Robotics
NVIDIA · University of California San Diego · Work completed during an internship at NVIDIA +1

Apr 10, 2026J/K move · Enter open · S save

AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly

Zhi Jing, Jinbin Qiao, Ouyang Lu, Jicong Ao+4

Fudan University† · Institute of Artificial Intelligence (TeleAI), China Telecom† · Tianjin University+3

3D Spatial Reasoning Robotic Assembly 3D VLMs VLMs for Robotics Object Pose Estimation

Open paper AskPDF arXiv

Spatial reasoning is a fundamental capability for embodied intelligence, especially for fine-grained manipulation tasks such as robotic assembly. Recent methods based on vision-language models (VLMs) largely rely on coarse 2D perception and struggle to perform accurate reasoning over complex 3D geometry. To address this limitation, we propose AssemLM, a spatial multimodal large language model for robotic assembly that integrates assembly manuals, point clouds, and textual instructions to predict task-critical 6D assembly poses with explicit geometric understanding. To bridge raw 3D perception and high-level linguistic reasoning, AssemLM employs a specialized point cloud encoder to extract fine-grained geometric and rotational features for accurate 3D spatial reasoning in assembly tasks. In addition, we introduce AssemBench, a large-scale benchmark for assembly-oriented spatial reasoning with over 900K multimodal samples and precise 6D pose annotations, extending evaluation from 2D grounding to full 3D geometric inference. Extensive experiments and real-robot evaluations demonstrate that AssemLM achieves state-of-the-art 6D pose reasoning performance and effectively supports fine-grained, multi-step assembly tasks in real-world settings. Code, models, and the AssemBench dataset will be made publicly available.

Abstract

Robotic Assembly

Explore similar work

AssemLM: A Spatial Reasoning Multimodal Large Language Model for Robotic Assembly

State-of-the-Art in Learning-by-Demonstration with Passive Observation for Industrial Assembly Automation

Generalizable Robotic Insertion with World Models