RollPlace: Improving Macro Placement via Monte Carlo Rollout Search

Qi Zhou; Guojun Liu; Guangzhi Qi; Ming Lu; Jiechu Liu; Zhongli Liu; Jianqun Yang; Xingji Li

cs.AIOct 5, 2026

RollPlace: Improving Macro Placement via Monte Carlo Rollout Search

Authors: Qi Zhou, Guojun Liu, Guangzhi Qi, Ming Lu, Jiechu Liu, Zhongli Liu, Jianqun Yang, Xingji Li

Organizations: Faculty of Computing, Harbin Institute of Technology, Harbin, 150001, China · School of Materials Science and Engineering, Harbin Institute of Technology, Harbin, 150001, China

Abstract

The application of Reinforcement Learning (RL) in Electronic Design Automation (EDA), particularly for chip placement, has attracted considerable attention in recent years. While existing machine learning (ML)-based approaches have achieved notable progress, they predominantly focus on generating optimal layouts in a single attempt, often producing solutions that require subsequent refinement. To address this limitation, we propose RollPlace, a novel and generalized macro placement framework. RollPlace adopts a two-stage optimization strategy: generating initial placement solutions via machine learning methods or heuristic-based strategies, and refining these layouts efficiently by adjusting specific macros derived from the initial stage. This strategy circumvents the sequential generation constraints inherent in traditional RL-based placement methods. Furthermore, RollPlace seamlessly integrates Monte Carlo Tree Search (MCTS) to balance exploration and exploitation, and employs a rollout mechanism for efficient local search. Extensive experiments on the ISPD 2005 benchmark demonstrate that RollPlace outperforms state-of-the-art methods. Additionally, end-to-end experimental results based on OpenROAD across 19 benchmarks show that RollPlace excels in multiple metrics. The proposed framework offers a robust and scalable solution for addressing the growing complexity of modern chip design challenges.

Figures & tables

Fig. 1: The general process of placing macros one by one in reinforcement learning. In each state

s_{t}

, the position of the

t

-th macro is determined according to the decision

a_{t}

given by the agent, and the environment gives the reward value

r_{t}

Fig. 2: RollPlace framework. In the first stage, multiple initial layouts are generated and a search tree is constructed using ML methods or others. In the second stage, candidate solutions are refined via a Monte Carlo rollout search, including selection, order exchange, simulation, rollout and update. The right black rectangle illustrates the process of simulation and rollout.

Panel of Figure 3 — Fig. 3: Macro adjustment. (a) $\boldsymbol{\pi}$ is the original solution, (b) $\boldsymbol{\pi}_{sub}$ is the subsequence which are going to be adjust, and (c) $\boldsymbol{\pi}-\boldsymbol{\pi}_{sub}$ keeps the original position.

Method	adaptec1	adaptec2	adaptec3	adaptec4	bigblue1	bigblue2	bigblue3	bigblue4
DREAMPlace	$13.80\pm 0.57$	$40.57\pm 3.06$	$62.34\pm 0.61$	$353.95\pm 43.76$	$7.38\pm 1.48$	$77.55\pm 9.63$	$86.53\pm 3.56$	$112.95\pm 4.03$
DeepPlace	$19.91\pm 2.13$	$203.51\pm 6.27$	$347.16\pm 4.32$	$311.86\pm 56.74$	$23.33\pm 3.65$	$11.38\pm 0.20$	$430.48\pm 12.18$	$433.90\pm 5.26$
MaskPlace	$7.56\pm 0.20$	$71.13\pm 3.93$	$106.00\pm 4.04$	$81.56\pm 3.34$	$3.16\pm 0.13$	$10.69\pm 0.88$	$82.55\pm 6.56$	$130.15\pm 10.94$
ChiPFormer	$7.40\pm 0.09$	$72.32\pm 5.41$	$80.05\pm 0.87$	$68.44\pm 1.29$	$3.08\pm 0.01$	$7.24\pm 0.37$	$89.36\pm 2.10$	$94.90\pm 2.50$
WireMask-EA	$6.23\pm 0.09$	$60.33\pm 3.67$	$63.04\pm 1.77$	$64.99\pm 1.17$	$2.17\pm 0.02$	$6.25\pm 0.24$	$66.79\pm 5.26$	$87.81\pm 3.30$
EfficientPlace	$5.99\pm 0.06$	$44.10\pm 1.64$	$60.29\pm 1.73$	$\boldsymbol{59.46\pm 1.60}$	$2.41\pm 0.03$	$\boldsymbol{4.88\pm 0.97}$	$77.90\pm 11.33$	$88.01\pm 2.71$

TABLE I: Comparisons of HPWL (

\times 10^{5}

). Lower HPWL values indicate better placement quality. Results for DeepPlace are sourced from [ 5 ] , while all other methods are evaluated by executing their respective open-source implementations with five distinct random seeds to ensure statistical reliability. For our proposed method, the minimum HPWL values achieved at

500

and

1000

optimization steps are recorded. Performance metrics are reported as mean values with standard deviations (mean

\pm

std). The best-performing results are highlighted in bold , while the second-best results are marked in brown .

Panel of Figure 5 — Fig. 4: Convergence analysis of the best HPWL values ( $\times 10^{5}$ ) for RollPlace, EfficientPlace and WireMask-EA across optimization steps. Solid lines denote mean values, while shaded regions represent standard deviation intervals derived from five independent trials with distinct random seeds.

Fig. 5: Visualization of intermediate placement results during the RollPlace search process on

adaptec2

benchmark. The red rectangle highlights macros that have shifted positions relative to the prior figure, while the blue rectangle indicates macros maintaining their positions.

Method		adaptec1	adaptec2	adaptec3	adaptec4	bigblue1	bigblue2	bigblue3	bigblue4
Traditional	DREAMPlace	$0.34$	$0.45$	$0.61$	$0.70$	$0.34$	$6.76$	$1.57$	$2.49$
AI-based	MaskPlace	$120$	$172$	$132$	$119$	$104$	$103$	$163$	$132$
	ChiPFormer	$16$	$21$	$22$	$21$	$19$	$19$	$22$	$29$
	WireMask-EA	$184$	$71$	$44$	$78$	$31$	$80$	$231$	$324$
	EfficientPlace	$71$	$116$	$132$	$236$	$83$	$30$	$255$	$123$
	RollPlace	$35$	$61$	$57$	$80$	$30$	$38$	$155$	$156$

TABLE II: Comparison of time consumption (in minutes) for macro placement across all algorithms. DREAMPlace leverages GPU acceleration to solve the placement problem. For time consumption metrics, a smaller value indicates superior performance.

Panel of Figure 8 — Fig. 6: Macro placement visualization for $adaptec2$ benchmark comparing RollPlace with five recent methods. Blue rectangles denote macros. The placement quality is evaluated through HPWL value, Lower HPWL values indicate superior performance.

Method	adaptec1	adaptec2	adaptec3	adaptec4	bigblue1	bigblue2	bigblue3	bigblue4
DREAMPlace	$3.78\pm 0.5$	${\color[rgb]{0.75,0.5,0.25}1.14\pm 0.08}$	$1.15\pm 0.02$	$2.06\pm 0.42$	$9.74\pm 2.01$	$3.64\pm 0.07$	$1.71\pm 0.08$	$1.83\pm 0.05$
MaskPlace	$2.81\pm 0.07$	$1.71\pm 0.12$	$1.40\pm 0.07$	$1.12\pm 0.04$	$4.42\pm 0.60$	$2.24\pm 0.14$	$1.22\pm 0.05$	$1.65\pm 0.11$
ChiPFormer	$4.91\pm 0.61$	$1.98\pm 0.12$	$1.14\pm 0.02$	$1.01\pm 0.01$	$1.81\pm 0.03$	$1.39\pm 0.12$	$1.30\pm 0.06$	$1.26\pm 0.09$
WireMask-EA	$1.07\pm 0.03$	$1.94\pm 0.15$	$1.11\pm 0.04$	$1.06\pm 0.04$	$1.42\pm 0.08$	$1.47\pm 0.06$	$1.25\pm 0.03$	$1.21\pm 0.05$
EfficientPlace	$1.06\pm 0.04$	$1.27\pm 0.11$	$1.08\pm 0.03$	$1.02\pm 0.02$	$1.52\pm 0.07$	$\boldsymbol{1.00\pm 0.04}$	$1.56\pm 0.32$	$1.11\pm 0.02$
RollPlace	$\boldsymbol{1.00\pm 0.04}$	$1.14\pm 0.22$	${\color[rgb]{0.75,0.5,0.25}1.04\pm 0.06}$	$\boldsymbol{1.00\pm 0.03}$	${\color[rgb]{0.75,0.5,0.25}1.02\pm 0.13}$	$1.15\pm 0.04$	${\color[rgb]{0.75,0.5,0.25}1.10\pm 0.09}$	${\color[rgb]{0.75,0.5,0.25}1.09\pm 0.06}$

TABLE III: Comparisons of congestion metrics. We standardize the best congestion values to

1

for a direct comparison. These metrics are calculated directly from the results in Table I using the RUDY algorithm without modifications. Note that DeepPlace results are excluded due to unavailable placement data. Lower values indicate superior congestion performance. The best-performing results are highlighted in bold , while the second-best results are marked in brown .

Panel of Figure 10 — Fig. 7: Comparison of HPWL relationship between macro placement and full placement on $adaptec3$ under different workflows. The green curve reflects HPWL trends across running steps in the independent workflow, whereas the orange curve captures HPWL trends in the proposed collaborative optimization process.

Method	adaptec1	adaptec2	adaptec3	adaptec4	bigblue1	bigblue3
DeepPlace	$8.28$	$12.33$	$23.24$	$23.40$	$14.10$	$46.86$
MaskPlace	$11.37\pm 0.13$	$13.29\pm 0.58$	$26.88\pm 1.50$	$25.11\pm 0.52$	$11.30\pm 0.81$	$54.94\pm 1.32$
ChiPFormer	$8.44\pm 0.54$	$12.05\pm 0.93$	$27.05\pm 1.14$	$23.76\pm 0.68$	$9.70\pm 0.26$	$52.90\pm 2.50$
WireMask-EA	$10.96\pm 0.30$	$13.55\pm 2.10$	$27.02\pm 1.50$	$25.96\pm 2.07$	$12.28\pm 2.67$	$57.90\pm 6.33$
EfficientPlace	$11.08\pm 0.52$	$12.98\pm 0.37$	$27.90\pm 1.12$	$26.85\pm 1.73$	$11.14\pm 0.51$	$53.63\pm 11.01$
DREAMPlace	${\color[rgb]{0.75,0.5,0.25}7.43}$	${\color[rgb]{0.75,0.5,0.25}8.42}$	${\color[rgb]{0.75,0.5,0.25}20.79}$	${\color[rgb]{0.75,0.5,0.25}17.90}$	${\color[rgb]{0.75,0.5,0.25}9.03}$	${\color[rgb]{0.75,0.5,0.25}30.71}$

TABLE IV: Comparisons of full placement HPWL results (

\times 10^{7}

). Lower HPWL values indicate superior performance. Data for DeepPlace [ 48 ] are obtained from [ 4 ] , while RollPlace results are generated via our full placement workflow. Results for other methods are derived from the workflow described above. The best-performing results are highlighted in bold , while the second-best results are marked in brown .

Panel of Figure 12 — Fig. 8: Final placement results of different methods for the $bp\_fe38$ benchmark. For key performance metrics, a smaller WireLength indicates better routing efficiency, while larger values of WNS and TNS reflect superior timing performance.

Explore similar work

Jun 8, 2026cs.AI

Order Matters: Unveiling the Hidden Impact of Macro Placement Sequences via Proxy-Guided LLM Evolution

Macro placement is a fundamental step in modern chip physical design, playing a crucial role in determining the solution quality of high-dimensional combinatorial optimization problems. Despite recent advancements in machine learning for spatial coordinate determination, the temporal dimension of placement sequencing remains largely governed by static heuristics. In this work, we demonstrate that the placement sequence is not merely a preprocessing step but a decisive factor in optimization, where suboptimal early decisions trigger irreversible domino effects that constrain the solution space. To harness this unexplored dimension, we propose \textbf{OrderPlace}, a proxy-guided LLM evolution framework for automatically discovering macro placement order strategies. Instead of relying on manually crafted heuristics such as area- or connectivity-based ordering, OrderPlace explores a broader space of code-level policies, ranging from static scoring metrics to dynamic physics-inspired mechanisms. To mitigate the prohibitive cost of evaluating sequences, we introduce a lightweight proxy evaluation mechanism that efficiently filters candidates using a deterministic greedy probe. Experimental results on the standard ISPD 2005 benchmarks demonstrate that OrderPlace discovers novel ordering strategies. Compared with WireMask-EA and the state-of-the-art method EGPlace, OrderPlace reduces wirelength by 34.04% and 14.08%, respectively.

Shibing Mo, Jing Liu, Jianchu Xu +1

the School of Artifcial Intelligence, Xidian University · the Guangzhou Institute of Technology, Xidian University.

Chip Design Layouts Order Matters Combinatorial Optimization Heuristics Impact Analysis

Ask PDF

Oct 5, 2026cs.AI

GPlaceRL: An Open-Source Graph Reinforcement Learning Framework for Detailed Placement

Reinforcement learning (RL) has emerged as a promising approach for placement optimization, particularly when combined with graph neural networks (GNNs) that capture circuit connectivity. However, most learning-based placement approaches focus on floorplanning, macro placement, or global placement, while detailed placement refinement remains relatively unexplored. In this paper, we present GPlaceRL, an open-source graph reinforcement learning framework for detailed placement refinement. GPlaceRL represents legalized placements as graphs and provides a modular environment for studying graph encoders, policy architectures, reward formulations, and local placement actions. To demonstrate the capabilities of GPlaceRL, we conduct a systematic evaluation of proximal policy optimization (PPO) policies with graph attention network (GAT) encoders in a per-design optimization setting. Across five placement benchmarks, the best greedy evaluation results achieve HPWL improvements ranging from

3.27\%

32.87\%

. The results highlight the importance of compact GAT architectures and flexible local action spaces for placement optimization. Overall, GPlaceRL provides a reproducible and extensible framework for systematic research on RL-based detailed placement refinement.

Pavlos Stoikos, Foteini Oikonomou, Christos Poulos +5

Department of Electrical and Computer Engineering, University of Thessaly, Volos, Greece · School of Computing Science, University of Glasgow, UK · Department of Electronic and Electrical Engineering, Trinity College Dublin, Ireland

Layouts Hierarchical Register Transfer Level Generation Optimal Sensor Placement

Ask PDF

Apr 28, 2026cs.AR

How Can Reinforcement Learning Achieve Expert-level Placement?

Chip placement is a critical step in physical design. While reinforcement learning (RL)-based methods have recently emerged, their training primarily focuses on wirelength optimization, and therefore often fail to achieve expert-quality layouts. We identify the reward design as the primary cause for the performance gap with experts, and instead of formalizing intricate processes, we circumvent this by directly learning from expert layouts to derive a reward model. Our approach starts from the final expert layouts to infer step-by-step expert trajectories. Using these trajectories as demonstrations or preferences, we train a model that captures the latent implicit rewards in expert results. Experiments show that our framework can efficiently learn from even a single design and generalize well to unseen cases.

Ruo-Tong Chen, Ke Xue, Chengrui Gao +7

State Key Laboratory of Novel Software Technology, Nanjing University, China · School of Artificial Intelligence, Nanjing University, China · Huawei Noah’s Ark Lab, China

Offline Reinforcement Learning Layouts Chip Design Single Trajectory

Ask PDF