In this work, we present a reinforcement learning (RL) based approach to designing parallel prefix circuits such as adders or priority encoders that are fundamental to high-performance digital design. Unlike prior methods, our approach designs solutions tabula rasa purely through learning with synthesis in the loop. We design a grid-based state-action representation and an RL environment for constructing legal prefix circuits. Deep Convolutional RL agents trained on this environment produce prefix adder circuits that Pareto-dominate existing baselines with up to 16.0% and 30.2% lower area for the same delay in the 32b and 64b settings respectively. We observe that agents trained with open-source synthesis tools and cell library can design adder circuits that achieve lower area and delay than commercial tool adders in an industrial cell library.

本文提出了一种基于强化学习的方法来设计高性能数字电路中的并行前缀电路，使用深度卷积强化学习代理在环境中进行训练，设计出Pareto优化现有基线的前缀加法电路，其面积和时延分别比现有基线低16.0％和30.2％。经实验表明，训练对公开源代码进行的综合工具和电池库的代理可以设计出优于商业工具加法器的电路。

使用深度强化学习优化并行前缀电路