Organizations: University of Chinese Academy of Sciences · Institute of Automation, Chinese Academy of Sciences · Institute of Information Engineering, Chinese Academy of Sciences · Tencent
Text-to-3D solid retrieval faces a fundamental representation gap: while natural language describes intrinsic shapes, Boundary Representation (B-rep) entangles precise geometry with arbitrary variations in pose, scale, and boundary encoding conventions. Existing methods either discretize B-reps at the cost of analytic precision, or learn features from topological graphs without explicit invariance guarantees. We propose a text-to-B-rep retrieval framework based on geometric equivalence classes and spectral invariants, mathematically eliminating representation ambiguities prior to cross-modal alignment. Formalizing translation, rotation, reflection, uniform scaling, and contour re-indexing via group actions, we exploit the planar-profile-and-extrusion structure of engineering solids to construct a unified spectral descriptor across cross-sectional regions, contour boundaries, and extrusion depths. On public benchmarks, our method achieves state-of-the-art recall performance across all evaluation metrics.
Figures & tables
Figure 1: Geometry is shared across B-rep encodings. (a) Changing the coordinate frame, contour starting point, or traversal direction changes the numerical encoding while preserving the solid. (b) The equivalent instances B1∼B2∼B3 share the same textual description and form a multi-positive set. Removing the bore changes the geometry and no longer matches the description.
Figure 2: Three ways to encode B-reps for text-to-CAD retrieval. (a) Rendered-view encoding converts B-reps into multi-view images for an image encoder. (b) Graph encoding represents faces and their adjacency as an attributed graph processed by a GNN. (c) Our group-theoretic encoding factors out geometric equivalence transformations and combines regional, extrusion, and boundary spectra into an invariant descriptor before geometry encoding, satisfying Φ(g⋅B)=Φ(B) for g∈G .
Figure 3: Product decomposition of a straight-extrusion unit. (a) A rounded B-rep solid with a through-hole, showing faces, edges, vertices, and oriented boundary loops (dashed arrows). (b) Its planar region Ω , boundary ∂Ω , and extrusion depth h ; ∂Ω sweeps along thickness h to form the lateral and inner-hole walls.
Figure 4: Spectral components on real B-reps. Four MMCAD-B parts are shown with regional Dirichlet modes on their top faces, solid modes um(x,y)sin(πz/h) on oblique interior cuts, and turning harmonics along their boundaries. The two rows use m=2,4 and k=2,4 ; the triangular frame uses k=3,6 . Colors indicate signed amplitudes normalized per field or boundary loop; zero harmonics remain neutral. Boundary panels include outer and hole contours, while the descriptor retains outer-contour Fourier magnitudes.
Figure 5: Invariance of the boundary spectrum to recording conventions. Changing the contour starting vertex from A to C or reversing traversal direction only changes phase or conjugates coefficients in the frequency domain; the Fourier magnitudes ∣Fk∣ remain identical.
Figure 6: Feature alignment under reordering on MMCAD-B. (a) Original and reordered B-rep features in a fixed PCA space; contours enclose 50%, 80%, and 95% probability mass, with overlap computed on a shared density grid. (b) Full-dimensional ℓ2 drift of normalized features; Coordinates denotes the sampled input-sequence diagnostic, and values below 10−6 share a display bin. (c) Original versus reordered text–B-rep matching scores; hexagon color gives pair counts and the dashed line indicates unchanged scores.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Full versus ablated descriptors. Arrows show the change in Expert R@5 from Full (blue) to each component removal (orange) with frozen text encoders. Orange point labels report R@5 (%); negative labels report changes in percentage points. Both panels use the same scale. Positive groups are recomputed for each variant.
Current token-sequence-based Large Language Models (LLMs) struggle to directly process 3D Boundary Representation (B-rep) models that contain complex geometric and topological information. To this end, we propose BrepLLM, the first multimodal framework that enables LLMs to directly parse and reason over raw B-rep data. BrepLLM adopts a two-stage training pipeline: cross-modal alignment pre-training and two-stage LLM fine-tuning. In the first stage, we design an adaptive UV sampling strategy to convert B-reps into graph representations that integrate geometric and topological information. Subsequently, we construct a hierarchical BrepEncoder to extract features from geometric elements (faces and edges) and topology, generating a global token and a sequence of node tokens. Then, via contrastive learning, we conduct an initial alignment between this global token and the text embeddings of a frozen CLIP text encoder (ViT-L/14). In the second stage, we integrate the pre-trained BrepEncoder into the LLM and employ a two-stage progressive strategy to align the sequence of node tokens: (1) training an MLP-based semantic mapping network that utilizes the prior knowledge of a 2D-VLM to align the B-rep representation to the 2D visual semantic space; (2) utilizing LoRA for parameter-efficient fine-tuning of the Q-Former and the LLM backbone network to achieve the final 3D-language generation capability. Furthermore, we construct the Brep2Text dataset, which contains 269,444 B-rep and text question-answer pairs. Experiments demonstrate that BrepLLM achieves SOTA performance on 3D object classification and captioning tasks. The project page is available at https://user-deng.github.io/BrepLLM/.
Liyuan Deng, Hao Guo, Yongkang Dai +5
Northwestern Polytechnical University · National University of Singapore · Hong Kong University of Science and Technology +1
Understanding and generating 3D objects as compositions of meaningful parts is fundamental to human perception and reasoning. However, most text-to-3D methods overlook the semantic and functional structure of parts. While recent part-aware approaches introduce decomposition, they remain largely geometry-focused, lacking semantic grounding and failing to model how parts align with textual descriptions or their inter-part relations. We propose DreamPartGen, a framework for semantically grounded, part-aware text-to-3D generation. DreamPartGen introduces Duplex Part Latents (DPLs) that jointly model each part's geometry and appearance, and Relational Semantic Latents (RSLs) that capture inter-part dependencies derived from language. A synchronized co-denoising process enforces mutual geometric and semantic consistency, enabling coherent, interpretable, and text-aligned 3D synthesis. Across multiple benchmarks, DreamPartGen delivers state-of-the-art performance in geometric fidelity and text-shape alignment.
Text-based retrieval of Computer-Aided Design (CAD) models is a critical yet underexplored task for the reuse of legacy industrial designs. Existing CAD repositories are typically searched using filenames or directories, which limits the efficiency, scalability, and accuracy of design retrieval. In this paper, we formally introduce text-to-CAD retrieval as a new cross-modal retrieval task, aiming to retrieve semantically relevant CAD models from large-scale databases given natural language queries. Leveraging paired text-CAD annotations from the Text2CAD dataset, we establish a practical benchmark for this task. To achieve text-based retrieval, we propose a unified framework that learns multi-modal CAD embeddings from both procedural sequences and geometric point clouds. Specifically, a sequence encoder captures the construction logic of CAD models, while a point encoder extracts explicit geometric features. A text encoder is used to learn semantic representations of textual queries. During training, we introduce a novel feature decoder that reconstructs masked sequence features via cross-attention with text and point features, encouraging implicit multi-modal alignment. At inference time, we remove this auxiliary decoder to enable efficient retrieval using concatenated sequence-point features. Our framework serves as a strong baseline for text-to-CAD retrieval and lays the foundation for downstream CAD generation paradigms, such as retrieval-augmented generation. The source code will be released.
Honghu Pan, Zibo Du, Daxiang Liu +2
School of Artificial Intelligence and Robotics, Hunan University · College of Computer Science and Software Engineer, Shenzhen University · Faculty of Science and Technology, University of Macau