Routing and switch placement are fundamental combinatorial optimization problems in chip design, requiring the joint optimization of routing topology and physical placement under strict structural, geometric and logical constraints. Existing approaches typically rely on carefully engineered heuristics that incorporate strong problem-specific biases to navigate the enormous space of possible designs. In this work, we introduce a hierarchical reinforcement learning framework for joint routing and switch placement at the level of logical communication routes. Starting from a minimal routing graph, our method progressively constructs increasingly expressive solutions through three coupled operations: switch expansion, switch placement, and route refinement. These operations preserve routing validity by construction, restricting exploration to feasible configurations where every communicating initiator-target pair has one assigned loop-free route. We explore the induced solution space using Gumbel Monte Carlo Tree Search, showing that neural-guided search substantially improves solution quality over non-learning optimization methods. Furthermore, pretraining across floorplans provides a strong initialization for fine-tuning on unseen instances.
Figures & tables
Figure 1: Overview of our routing and switch-placement method. Gumbel MCTS guides repeated switch expansion, Hanan-grid placement, and route refinement while preserving routing feasibility.
Figure 2: Route-node conversion. Each communication pair traversing an edge is represented explicitly by a route node.
Figure 3: Extended Hanan grid induced by initiators, targets, and blockage corners.
Figure 4: Switch expansion and route refinement. Expansion of a switch (leftmost yellow node in the first part of the figure) introduces a second switch and exposes four local routing alternatives for each affected communication pair. Route refinement selects one alternative, restoring a unique local route while leaving the remainder of the communication path unchanged.
Method
1
2
3
4
5
6
7
8
9
10
11
12
Heuristic
11.902
11.255
15.349
11.330
14.847
11.126
8.090
10.301
11.292
14.615
14.741
13.418
Random search
18.706
20.799
24.042
17.420
23.629
19.310
12.672
18.574
18.663
24.931
22.584
20.775
Genetic algorithm
14.382
13.999
16.854
12.622
17.970
13.107
8.224
11.954
14.152
15.799
16.478
16.504
PPO-EWMA
12.166
11.484
15.670
11.576
14.717
10.960
8.595
10.352
11.142
13.329
14.151
13.361
Gumbel MCTS
11.333
10.880
14.274
9.784
13.926
9.918
7.739
9.897
10.820
13.028
13.995
13.361
Table 1: Objective values on the 24 training floorplans (lower is better). Best results are shown in bold and second-best results are underlined.
Figure 5: Comparison between optimization from scratch and fine-tuning from a policy pretrained on the 24 training floorplans, evaluated on held-out floorplans. Each curve shows the mean over three independent runs, and the shaded region indicates one standard deviation. The objective is shown on a logarithmic scale. For readability, the time axis is truncated once all methods are within 1% of their respective best objective values. PPO-EWMA is shown in orange and Gumbel MCTS in blue. Solid lines correspond to fine-tuning from the pretrained policy, while dotted lines correspond to optimization from scratch.
Department of Electrical and Computer Engineering, University of Thessaly, Volos, Greece · School of Computing Science, University of Glasgow, UK · Department of Electronic and Electrical Engineering, Trinity College Dublin, Ireland
Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China · School of Materials Science and Engineering, Harbin Institute of Technology, Harbin, 150001, China
State Key Laboratory of Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Huawei Noah’s Ark Lab, China