Topology-Aware Reinforcement Learning for CAD Decomposition toward All-Hexahedral Meshing
Authors: Jiaming Peng, Zhihao Zheng, Rui Wang, Qinglin Wang, Jing Xiao, Jie Liu, Na Lei, Xinhai Chen
Organizations: Laboratory of Digitizing Software for Frontier Equipment, the National Key Laboratory of Parallel and Distributed Computing, and the College of Computer Science and Technology, National University of Defense Technology, Changsha 410073, China. · the School of Data Science and Artificial Intelligence, Wenzhou University of Technology, Wenzhou 325000, China. · the School of Mechanical Engineering and Intelligent Manufacturing, Ningbo University, Ningbo 315211, China. · the School of Software, Dalian University of Technology, Dalian, China.
Automatic hexahedral meshing for complex CAD models has long been regarded as the "holy grail" of mesh generation. The fundamental challenge arises from feature misalignment, where orthogonal or interacting features coexist, making direct hexahedral meshing infeasible and requiring decomposition into simpler, meshable blocks. While existing decomposition methods, including frame-field pipelines and learning-based techniques, have reduced manual intervention, they can remain sensitive to geometric and topological complexity and may fail to produce valid decompositions for challenging feature configurations. Specifically, their action spaces are typically restricted to surface-based splitting, which often yields invalid or unmeshable decompositions. To address these limitations, we propose DecompRL, a deep reinforcement learning framework for CAD decomposition toward hexahedral meshing. We design a set of topology-aware internal-splitting actions that leverage both global topological structure and local geometric property to effectively separate misaligned features. Furthermore, we introduce geometric rewards together with meshability feedback to guide the learning process toward valid decompositions and high-quality meshes. Extensive experiments on ABC, Fusion360, NIST, and DeepCAD show that DecompRL achieves the highest success rate among the evaluated learning-based methods on three of the four benchmarks, while maintaining competitive mesh quality. The measured LMFF-to-DecompRL runtime ratio reaches up to two orders of magnitude; this compares the full LMFF procedure with DecompRL decomposition alone, excluding subsequent mesh generation. The largest SR improvement is observed on Fusion360, where DecompRL reaches a success rate of 59.6%.
Figures & tables
Fig. 1: Comparison of representative CAD decomposition and hexahedral meshing pipelines.
Fig. 2: Representative hexahedral meshing results produced by DecompRL across diverse CAD models.
Fig. 3: Overview of DecompRL. (a) Starting from the active solid Bt , the environment constructs a typed candidate pool Ct=G(Bt) together with the state representation st=ψstate(Bt) , which combines features extracted by a pretrained B-Rep encoder. (b) The generated candidates are encoded into the candidate set Xt , filtered by the validity mask mt , and assembled into the policy observation (st,Xt,mt) for candidate selection. In the illustrated example, candidate ct4 does not yield a valid split, so its mask entry is set to mt,4=0 . (c) The selected geometric action at is executed on Bt , producing child solids Child(Bt,at) that are pushed into the decomposition frontier for later decisions. (d) Reward terms and an aggregator over child values are combined to form the tree target used to update the actor and critics.
Fig. 4: Local frontier expansion in the decomposition tree. A selected split can generate multiple child solids, which reenter the decomposition frontier for later decisions.
Fig. 5: Planar, symmetry, O-grid, feature gap, and rotation splits.
Fig. 6: Feature-gap candidate construction on a synthetic example. Orthogonal rectangular- and cylindrical-hole features are grouped, their extents are projected along the separation direction, and a plane is instantiated at the gap midpoint. Transparent exteriors reveal the hole walls. The example illustrates geometric partitioning; hexahedral meshability is not evaluated here.
Fig. 7: Reward-guided geometric preferences. (a) and (b) compare dihedral-angle configurations for qE . (c) shows an extreme geometry case penalized by qG . (d) shows a case in which fewer internal loops improve qL .
Fig. 8: Shared-interface conformity in the MAMBO M1 mesh generated with SALOME. (a) Selected interface patch. (b) Matching subdivisions with five intervals per side. (c) Shared patch mesh with 25 quadrilaterals and 36 nodes. (d) Two adjacent hexahedra share the four nodes of face q .
DecompRL
DDPG
TD3
PPO
Hidden dimension
256
256
256
256
Learning rate
1×10−4
1×10−4
1×10−4
1×10−4
Batch size
128
128
128
128
Target update τ
0.005
0.005
0.005
–
Grad. norm clip
1.0
1.0
1.0
1.0
Discount factor γ
0.95
0.95
0.95
0.95
TABLE I: Training configuration and parameter counts of the compared methods.
Fig. 9: Representative decomposition results produced by DecompRL. Colors denote child solids generated by the selected splits.
Fig. 10: Qualitative comparison of decomposition and hexahedral meshing on six representative CAD models. The upper half shows decomposition results, where different colors denote different sub-blocks. The lower half shows hexahedral meshing results colored by scaled Jacobian, where blue indicates better mesh quality and red indicates failure to generate hexahedral meshes.
Fig. 11: Case 1 (ABC 00013257). (a) Original CAD. (b) Original 11-leaf frontier after six accepted actions. (c) Later 49-body diagnostic partition, shown as geometry. (d) Nine unmeshed bodies. The follow-up mesh remained incomplete. Colors distinguish bodies, not quality classes.
Fig. 12: Case 2 (ABC 00012519). (a) Original positive-volume source component. (b) Later 28-body diagnostic partition. (c) Partial mesh on 25 bodies. (d) The three unmeshed feature caps, shown together. The newly introduced central interfaces remain unverified. Colors distinguish bodies, not quality classes.
Dataset
Baseline
SR difference
Raw p
Holm-adjusted p
ABC
PPO
+1.6
0.4521
0.9043
Fusion360
PPO
+12.6
6.77×10−5
6.77×10−4
NIST
PPO
+2.1
0.0261
0.1307
DeepCAD
DDPG
−1.3
0.1129
0.4517
TABLE II: SR comparisons against the RL baseline with the highest mean SR on each dataset. Differences are DecompRL minus the baseline, in percentage points. Holm correction covers all 12 RL-baseline comparisons, not only the four shown.
Method
ABC
Fusion360
NIST
DeepCAD
SR (%)
J > 0.8 (%)
minJs
R
SR (%)
J > 0.8 (%)
minJs
R
SR (%)
J > 0.8 (%)
minJs
R
SR (%)
J > 0.8 (%)
minJs
R
PPO
39.0 ± 1.3
89.0 ± 1.1
0.5 ± 0.0
1.4 ± 0.1
47.0 ± 0.9
85.8 ± 0.4
0.5 ± 0.1
3.4 ± 0.2
37.5 ± 0.6
80.1 ± 0.1
0.3 ± 0.1
2.7 ± 0.5
75.8 ± 0.1
85.2 ± 1.0
0.6 ± 0.0
3.1 ± 0.2
DDPG
32.0 ± 1.8
86.7 ± 1.7
0.4 ± 0.1
1.9 ± 0.1
44.0 ± 0.7
89.2 ± 0.4
0.6 ± 0.1
4.1 ± 1.0
34.3 ± 0.3
71.7 ± 0.6
0.3 ± 0.0
3.3 ± 0.8
77.3 ± 0.5
89.0 ± 0.1
0.5 ± 0.1
3.7 ± 0.3
TD3
36.5 ± 2.2
90.1 ± 2.0
0.4 ± 0.1
1.6 ± 0.3
43.5 ± 0.7
88.9 ± 0.0
0.5 ± 0.1
3.3 ± 0.5
32.5 ± 0.2
83.5 ± 0.7
0.4 ± 0.0
2.5 ± 1.0
71.2 ± 0.1
88.4 ± 0.6
0.8 ± 0.0
3.2 ± 0.7
Zhang et al. [ 21 ]
25.0 ± 2.6
84.9 ± 2.4
0.3 ± 0.0
1.2 ± 0.0
34.0 ± 0.8
84.4 ± 0.3
0.4 ± 0.1
4.3 ± 0.2
27.5 ± 1.0
86.8 ± 0.5
0.4 ± 0.0
3.3 ± 0.9
65.6 ± 0.3
82.1 ± 0.5
0.6 ± 0.1
2.6 ± 0.2
DecompRL (ours)
40.6 ± 2.9
91.6 ± 2.9
0.5 ± 0.1
3.2 ± 0.3
59.6 ± 0.7
88.2 ± 0.2
0.6 ± 0.1
5.2 ± 0.5
39.6 ± 0.0
87.5 ± 0.8
0.4 ± 0.0
3.9 ± 0.6
76.0 ± 0.9
88.7 ± 0.2
0.8 ± 0.1
3.5 ± 0.4
TABLE III: Quantitative comparison across methods and datasets under the unified evaluation setup. Values are reported as mean ± std over seeds 42, 43, and 44; higher is better for all reported metrics.
Fig. 13: Aggregation and action-space ablations of DecompRL. (a) compares sum, mean, and max aggregation over child values on ABC; values are mean SR over seeds 42, 43, and 44. (b) reports action-space ablation on Fusion360 ( N=200 ); bar labels give SR and in-bar labels show the SR change in percentage points relative to Full DecompRL. The y-axis is truncated below 45% to resolve differences among high-SR variants.
Fig. 14: Complete-model success rate during training on a fixed set of 200 Fusion360 CAD models. Each curve shows one run per method. A model counts as successful when hexahedral meshing completes for the entire model.
Fig. 15: Archived decompositions from a separate historical training run, redrawn in FreeCAD. The top row shows ABC 00006119 and the bottom row shows ABC 00026545. The left and right boxes show the earlier and later episodes, respectively, with the episode number above each pair of views. Each pair shows the partition from above and from the opposite lower side. Colors distinguish solids within each view and do not track solids across episodes.
Dataset
DecompRL
Greedy
Random-valid
Max. adjusted p
ABC
40.6
16.0
14.7±1.6
<0.01
Fusion360
59.6
16.5
17.8±0.6
<0.01
NIST
39.6
11.5
14.8±2.0
<0.01
DeepCAD
76.0
26.5
20.2±1.4
<0.001
TABLE IV: Non-learning selector comparison. All SR values are percentages. The last column is the maximum Holm-adjusted McNemar p -value across three DecompRL seeds against Greedy, separately for each dataset.
Variant
SR (%)
J > 0.8 (%)
minJs
Mean Reward
Main Component Ablations
Full Model
40.62 ± 2.91
91.64 ± 2.88
0.46 ± 0.09
3.22 ± 0.30
w/o Tree Bellman
31.88 ± 2.65
87.00 ± 1.12
0.43 ± 0.17
3.04 ± 0.02
w/o Advanced Actions
30.00 ± 1.77
91.63 ± 0.86
0.36 ± 0.04
2.94 ± 0.13
Early-fusion policy (same candidate pool)
30.62 ± 4.42
86.81 ± 0.12
0.38 ± 0.04
2.93 ± 0.20
Reward Component Ablations
TABLE V: Ablation results for the architectural and reward components of DecompRL. Each variant is trained with three random seeds; values are mean ± std. The Full Model row corresponds to DecompRL in Table III ; minor differences are due to rounding. Boldface marks the best value in each primary endpoint column.
Fig. 16: Reward-weight sensitivity of DecompRL. Results are reported across SR, J > 0.8, minJs , and Mean Reward. Rows correspond to reward terms; the center band gives the default value once, and the outer columns give signed changes induced by 0.7 × and 1.3 × scaling. Colors are normalized separately for each metric.
Configuration
λd
γ
SR (%)
Depth
Steps
High-AR (%)
Stronger decay
0.80
0.95
56.5
2.5
5.2
3.5
Default
0.90
0.95
59.6
2.8
5.77
4.1
Lower Bellman discount
0.90
0.90
58.2
2.7
5.5
3.8
No depth decay
1.00
0.95
47.2
3.8
7.2
14.6
TABLE VI: Discount sensitivity on Fusion360 under an 8-step budget. Depth and steps are model-level means; high-AR denotes the aggregate percentage of accepted leaf blocks with high aspect ratios. The default configuration corresponds to the main-paper baseline.
Fig. 17: Runtime comparison with different timing scopes. (a) compares DecompRL decomposition time, excluding subsequent mesh generation, with the complete measured LMFF procedure. (b) reports their runtime ratio, labeled Speedup; it is not an end-to-end meshing speedup.
Fig. 18: Refined and coarse hexahedral meshes generated from the same CAD model after decomposition.
Dataset
Time/sample (s)
Steps
Leaves
Valid ratio
ABC
26.745
5.665
5.380
0.184
Fusion360
24.594
5.770
5.979
0.190
NIST
25.303
6.060
5.944
0.165
DeepCAD
6.087
1.960
1.772
0.108
TABLE VII: Per-dataset decomposition diagnostics of DecompRL under the fixed N=200 setting: decomposition time per sample (excluding subsequent mesh generation), decomposition length (steps), exported leaf blocks, and candidate validity ratio.
Generating high-quality meshes for arbitrary geometries remains a fundamental bottleneck in computational engineering, often demanding heuristic tuning and semi-manual workflows. In this paper, we introduce Dmsh, a first fully automated reinforcement learning pipeline that unifies geometric decomposition and quadrilateral mesh generation within a single learning-based framework. Dmsh decomposes the problem through three coordinated agents handling topology simplification, geometric regularization, and mesh generation. The meshing process is formulated as a Markov Decision Process and solved using a parametric Soft Actor-Critic architecture with decoupled critics, enabling efficient exploration of a hybrid discrete-continuous action space. A curriculum learning strategy ensures scalability from simple domains to highly complex geometries, suppressing seed variance. By design, the recursive decomposition enables parallel meshing of subregions, yielding globally conforming all-quadrilateral meshes without post hoc correction. Across a wide range of benchmarks, Dmsh consistently outperforms existing methods in automation, robustness, and mesh quality, establishing a new paradigm for learning-based mesh generation.
A quadrilateral block decomposition of a planar domain is judged by whether it is complete, whether its elements are well shaped, and how many of its vertices are irregular. The last has a provable floor: the discrete Gauss-Bonnet identity enforces a lower bound on the total vertex irregularity of any all-quadrilateral mesh of a given domain purely based on its topology and corner angles. We train a reinforcement learning agent to build decompositions that reach this bound, which we call par. It acts directly on the mesh's half-edge data structure through local edits, with a policy network whose convolutions follow the mesh's own connectivity, so it applies unchanged to domains larger than any seen in training. The reward targets the floor directly, and it is sparse: random play reaches it on no domain with more than eight sides. We overcome this exploration barrier via behaviour cloning on optimal meshes that are trivial to construct, walked backward into demonstrations, before training it with PPO. On 96 held-out domains the agent produces an all-quadrilateral mesh on every one, a usable one on 95.7 on average, and a provably optimal one on 90; Gmsh's strongest configuration at the same element count completes 51, is usable on 38 and optimal on none, and even at three to fourteen times the elements never produces a more regular mesh. On 64 domains twice the training size the agent completes all, is usable on 62, and keeps a median excess over par below one against Gmsh's 39 at the same element count.
Decomposing a polyhedral solid into a few well-shaped hexahedral blocks is a long-standing open problem in mesh generation. We cast it as a sequential decision problem on a 3-dimensional generalised map: an agent splits edges, faces and volumes with three insert operations until every cell is a hexahedron and the mesh passes a quality threshold, and a policy that convolves over the map's involutions chooses each insert. A successful decomposition takes over a hundred decisions, and a uniformly random policy never finds one. We clone a policy from constructed decompositions and improve it by expert iteration. On held-out chamfered solids that share no boundary topology with the cloning data, the final policy solves an expected 94.4% in 20 samples, and expert iteration raises the best-of-20 solve rate on large chamfered solids from 0.203 to 0.355. At matched compute, PPO, GRPO and VinePPO updates all make the cloned policy worse.
Arjun Narayanan, Per-Olof Persson
University of California, Berkeley · Department of Mathematics, University of California, Berkeley