We propose MeshOn, a method that finds physically and semantically realistic compositions of two input meshes. Given an accessory, a base mesh with a user-defined target region, and optional text strings for both meshes, MeshOn uses a multi-step optimization framework to realistically fit the meshes onto each other while preventing intersections. We initialize the shapes' rigid configuration via a structured alignment scheme using Vision-to-Language Models, which we then optimize using a combination of attractive geometric losses, and a physics-inspired barrier loss that prevents surface intersections. We then obtain a final deformation of the object, assisted by a diffusion prior. Our method successfully fits accessories of various materials over a breadth of target regions, and is designed to fit directly into existing digital artist workflows. We demonstrate the robustness and accuracy of our pipeline by comparing it with generative approaches and traditional registration algorithms.
Figures & tables
Figure 2: Digital artists often load shapes from pre-existing libraries of meshes that include additional information like animation rigs, skinning weights, texture maps and more. Generative shape editing methods like Instant3dit [ 4 ] discard this information, performing global changes to the shapes and merging them together. MeshOn is designed to fit exactly within this common artistic pipeline, and perfectly preserves all pre-existing information on the input meshes.
Figure 3: We compose the two meshes through a multi-step optimization pipeline: from an initialization obtained with a Vision-to-Language Model ( subsection 3.5 ), we start by obtaining a tight fit containing intersections ( subsection 3.1 ), which we then resolve ( subsection 3.2 ). After finetuning the fit to obtain the best possible rigid fit ( subsection 3.3 ), we improve it further by allowing small deformations in the accessory object ( subsection 3.4 ).
Figure 4: MeshOn is capable of fitting a wide variety of accessories on a range of different meshes. Our method handles challenging spatial relationships such as sliding glasses along a head to align with the ears, positioning hats and helmets to conform to head curvature, wrapping bands around articulated limbs, and fitting objects along long, curved surfaces. These examples demonstrate the flexibility of our fitting trajectory and its ability to adapt accessories to widely varying shapes and poses.
Figure 5:
Figure 6: MeshOn is capable of fitting the accessory onto the base mesh in a way that considers material properties. Instead of having to specify the numeric material parameters, artists can provide material guidance through text prompts, that get combined with rendered images and interpreted by a VLM to output specific elastic parameters.
Figure 7: MeshOn uses a multi-step process, where each of them is critical for achieving a desirable mesh to mesh composition result. Omitting any of the steps yields a semantically (two left results) or physically (two center results) unplausible solutions. Utilizing Step 4 further improves the result (second from the right), allowing the asset to deform and better fit the base mesh (rightmost).
Figure 8:
Figure 9: Our method is moderately robust to different initialization configurations, although it can fail to converge to the desired output in very adversarial cases (leftmost and rightmost). In subsection 3.5 , we introduce a VLM initialization strategy that avoids these cases.
Figure 10: Region-controlled composition. (a) Given four distinct user-selected target regions, MeshOn places a ring precisely on each selected region, demonstrating explicit user control over the fitting process. (b) Because the method adheres strictly to the selected region, asset placement is sensitive to the region definition; for example, glasses sit naturally on the ears only if a small portion of the ear region is included in the selection.
Figure 11: Unlike ours (left), classical registration algorithms like ICP contain no semantic guidance: therefore, they will produce unrealistic configurations; e.g., glasses that sit on the eyes upside down. These algorithms are also not designed to avoid intersections; therefore, they will produce many of them (see blowups). See supplementary materials for details of these methods.
Alignment Baselines
Metric
Instant3Dit
Best ICP
Best FGR
RANSAC+ICP
RANSAC+FGR
Ours
CLIP ( ↑ )
0.311
0.341
0.333
0.331
0.321
0.356
CLIP-IQA ( ↑ )
0.559
0.586
0.609
0.608
0.610
0.604
VQA ( ↑ )
65.62
73.71
73.97
73.72
73.68
74.49
Ablation Study
Table 1: Quantitative evaluation using CLIP, CLIP-IQA, and VQA scores. Top: comparison against alignment baselines. Bottom: ablation study of our multi-step pipeline. Scores are averaged over per-example best renderings (best of 4); higher is better.
Ablation Study
Metric
Best ICP
Best FGR
RANSAC+ICP
RANSAC+FGR
Ours
Intersecting faces ( ↓ )
628
660
544
434
0
Max penetration ( ↓ )
0.0197
0.0237
0.0243
0.0237
0
Table 2: Penetration statistics averaged over ten experiments. We report the total number of intersecting faces and the maximum penetration depth. Our method produces intersection-free compositions (all zeros).
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: We show a comparison of runtime and GPU memory consumption for calculating a single distance measure d using our GPU-optimized BVH structure. We run the calculation with varying resolutions of the base mesh (visualized on top of each mesh), without using any user defined masks. We use a target mesh with a constant size of 8646 vertices and 17288 faces.
User Study Results
Statistic
Instant3Dit
Best ICP
Best FGR
RANSAC+ICP
RANSAC+FGR
Ours
Average Preference (%) ↑
2.81
3.13
0.63
0.00
0.63
92.81
Minimum (%)
0.00
0.00
0.00
0.00
0.00
81.25
Maximum (%)
15.63
12.50
3.13
0.00
3.13
100.00
Appendix
Table 3: User study results comparing six methods for the 3D wearable compositing task. Each entry reports the percentage of participant selections received by each method across all evaluation prompts (32 participants per prompt). We report the average preference percentage, as well as the minimum and maximum percentage achieved by each method on any individual example.
We present MeshLoom, a feed-forward registration network that directly reconstructs vertex deformations across mesh sequences. Our approach advances non-rigid registration beyond existing models, which are typically constrained by costly per-instance optimization, narrow object categories, pairwise-only inputs, or merely intermediate outputs. The network is simple and efficient, registering multiple meshes within seconds. At its core lies a topology-aware encoder--decoder design. Specifically, we first introduce a topology-aware point representation that encodes the anchor (reference) mesh's topology into its per-vertex features. This representation strengthens the network's understanding of the anchor-mesh geometry and disambiguates points that are Euclidean-close yet geodesically distant. We then propose a multi-modal encoder that fuses this anchor-mesh representation with complementary cues from each frame, such as shape latents and image features. These multi-source signals are compressed into a compact global motion embedding that captures dense inter-frame correspondence. A lightweight decoder then queries this global embedding with the anchor-mesh point representation, retrieving per-vertex deformations at target timestamps. Through extensive experiments across diverse motions and object categories, we show that MeshLoom achieves state-of-the-art results on non-rigid registration. In addition, we find that our global embedding-then-query paradigm naturally enables the network to generate deformations at intermediate timestamps, which extends MeshLoom to motion interpolation and mesh morphing. Project page: https://meshloom.github.io/ .
Recent breakthroughs in 3D generation have enabled the synthesis of high-fidelity individual assets. However, generating 3D compositional objects from single images--particularly under occlusions--remains challenging. Existing methods often degrade geometric details in hidden regions and fail to preserve the underlying object-object spatial relationships (OOR). We present a novel framework Interact3D designed to generate physically plausible interacting 3D compositional objects. Our approach first leverages advanced generative priors to curate high-quality individual assets with a unified 3D guidance scene. To physically compose these assets, we then introduce a robust two-stage composition pipeline. Based on the 3D guidance scene, the primary object is anchored through precise global-to-local geometric alignment (registration), while subsequent geometries are integrated using a differentiable Signed Distance Field (SDF)-based optimization that explicitly penalizes geometry intersections. To reduce challenging collisions, we further deploy a closed-loop, agentic refinement strategy. A Vision-Language Model (VLM) autonomously analyzes multi-view renderings of the composed scene, formulates targeted corrective prompts, and guides an image editing module to iteratively self-correct the generation pipeline. Extensive experiments demonstrate that Interact3D successfully produces promising collsion-aware compositions with improved geometric fidelity and consistent spatial relationships.
Hui Shan, Keyang Luo, Ming Li +4
Zhejiang University, China · Shanghai Innovation Institute, China · Westlake University, China +4
We present MeshTailor, the first mesh-native generative framework for synthesizing edge-aligned seams on 3D surfaces. Unlike prior optimization-based or extrinsic learning-based methods, MeshTailor operates directly on the mesh graph, eliminating projection artifacts and fragile snapping heuristics. We introduce ChainingSeams, a hierarchical serialization of the seam graph that orders chains from global structural cuts down to local details in a coarse-to-fine manner, and a dual-stream encoder that fuses topological and geometric context. Leveraging this hierarchical representation and dual-stream vertex embeddings, our MeshTailor Transformer utilizes an autoregressive pointer layer to trace seams vertex-by-vertex within local neighborhoods. Extensive evaluations show that MeshTailor produces more coherent and structurally regular seam layouts compared to recent optimization-based and learning-based baselines.