Geometric 2D Scene Graph Generation

Christoph Jahn; Urs Waldmann; Bastian Goldluecke

cs.CVJul 24, 2026

Geometric 2D Scene Graph Generation

Authors: Christoph Jahn, Urs Waldmann, Bastian Goldluecke

Organizations: 1Mercedes-Benz AG, Sindelfingen, Germany · Department of Computer and Information Science, University of Konstanz, Konstanz, Germany

Abstract

In production processes for consumer products, assembly instructions are essential not only for planning but also for executing the production process. Likewise in robotics, it is crucial for an assembly robot to understand how components fit together and can be assembled. To facilitate these tasks, we contribute a method for constructing scene graphs to represent and characterize assembly relationships between components. Our approach does not rely on semantic data and is capable of handling a very small dataset. To realize this, the output of a Faster R-CNN model is used to create geometric representations, which are then processed by a transformer architecture to generate an adjacency matrix. This matrix serves as input to a Siamese network that uses message passing based on an attentional graph convolutional network (aGCN) architecture to characterize the connections between the components. We validate our method on a study dataset of toy model components which can be assembled into transportation vehicles.

Explore similar work

Linkify: Learning from Interface-Augmented Assembly Graphs
Jul 1, 2026Anushrut Jignasu, Daniele Grandi3D Models Geometric Reconstruction
AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects
May 13, 2026Danrui Li, Jiahao Zhang, Bernhard Egger +43D Models Manufacturing
GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning
Date pendingLong Xu, Zhiqi Zhang, Mi Yan +8Indoor Scene Generation Articulated Objects

Jul 1, 2026 · cs.CVJ/K move · Enter open · S save

Linkify: Learning from Interface-Augmented Assembly Graphs

Anushrut Jignasu, Daniele Grandi

Iowa State University, USA · Autodesk Research, USA

We present Linkify, a framework for learning from interface-augmented assembly graphs to enable context-aware part retrieval in mechanical assemblies. While recent generative AI methods for CAD have focused largely on isolated parts or monolithic assemblies, the rich geometric information at the interfaces between parts, where function is realized, remains underexplored. We address this gap by recomputing high-fidelity interface geometry for the Fusion 360 Gallery Assembly dataset, correcting missing and erroneous contacts, and generating point-cloud representations of local contact regions. Using this data, we construct assembly graphs whose nodes encode part geometry and whose edges encode interface geometry via a pretrained point-cloud encoder. On top of this representation, we train a Graph Attention Network based on GATv2 to solve a masked part prediction task: given an assembly with one part held out, the model predicts the class of the missing component from a large vocabulary of geometrically clustered parts, thereby approximating a realistic part-retrieval scenario. Compared to non-graph baselines such as logistic regression and k-nearest neighbors operating on aggregated node features, Linkify achieves higher Top-K accuracy and F1 scores. Ablation studies on graph connectivity, edge attributes, and attention mechanisms demonstrate that accurate contact computation and dynamic attention over interfaces are critical for performance. Our corrected interface dataset and training pipeline, released publicly, provide a foundation for future interface-aware models for assembly retrieval, validation, and generative design.

3D Models Geometric Reconstruction Generative Artificial Intelligence

Open paper Ask about it PDF

Abstract

Explore similar work

Linkify: Learning from Interface-Augmented Assembly Graphs

AssemblyBench: Physics-Aware Assembly of Complex Industrial Objects

GIF: Agentic Generation of Interactive and Functional Object Compositions for Robot Learning