Recently, successful applications of reinforcement learning to chip placement
have emerged. Pretrained models are necessary to improve efficiency and
effectiveness. Currently, the weights of objective metrics (e.g., wirelength,
congestion, and timing) are fixed during pretraining. However, fixed-weighed
models cannot generate the diversity of placements required for engineers to
accommodate changing requirements as they arise. This paper proposes flexible
multiple-objective reinforcement learning (MORL) to support objective functions
with inference-time variable weights using just a single pretrained model. Our
macro placement results show that MORL can generate the Pareto frontier of
multiple objectives effectively.

本文提出一种灵活的多目标强化学习方法 (MORL)，使用单个预先训练的模型来支持具有推理时间变量权重的目标函数，并成功地将其应用于芯片布局中，有效地生成多个目标的 Pareto 前沿。

芯片布局的灵活多目标强化学习

Flexible Multiple-Objective Reinforcement Learning for Chip Placement

In this work, we present a learning-based approach to chip placement, one of
the most complex and time-consuming stages of the chip design process. Unlike
prior methods, our approach has the ability to learn from past experience and
improve over time. In particular, as we train over a greater number of chip
blocks, our method becomes better at rapidly generating optimized placements
for previously unseen chip blocks. To achieve these results, we pose placement
as a Reinforcement Learning (RL) problem and train an agent to place the nodes
of a chip netlist onto a chip canvas. To enable our RL policy to generalize to
unseen blocks, we ground representation learning in the supervised task of
predicting placement quality. By designing a neural architecture that can
accurately predict reward across a wide variety of netlists and their
placements, we are able to generate rich feature embeddings of the input
netlists. We then use this architecture as the encoder of our policy and value
networks to enable transfer learning. Our objective is to minimize PPA (power,
performance, and area), and we show that, in under 6 hours, our method can
generate placements that are superhuman or comparable on modern accelerator
netlists, whereas existing baselines require human experts in the loop and take
several weeks.

本文提出了一种基于学习的芯片布局方法，使用强化学习将芯片画布上芯片网络节点的摆放视作问题，并训练代理人生成以往未见芯片块的优化布局，最终旨在最小化能耗、性能与面积（PPA）。实验结果表明，该方法可在 6 小时内生成可与人工专家相媲美的布局。