In the rapidly evolving semiconductor industry, where research, design,
verification, and manufacturing are intricately linked, the potential of Large
Language Models to revolutionize hardware design and security verification is
immense. The primary challenge, however, lies in the complexity of hardware
specific issues that are not adequately addressed by the natural language or
software code knowledge typically acquired during the pretraining stage.
Additionally, the scarcity of datasets specific to the hardware domain poses a
significant hurdle in developing a foundational model. Addressing these
challenges, this paper introduces Hardware Phi 1.5B, an innovative large
language model specifically tailored for the hardware domain of the
semiconductor industry. We have developed a specialized, tiered dataset
comprising small, medium, and large subsets and focused our efforts on
pretraining using the medium dataset. This approach harnesses the compact yet
efficient architecture of the Phi 1.5B model. The creation of this first
pretrained, hardware domain specific large language model marks a significant
advancement, offering improved performance in hardware design and verification
tasks and illustrating a promising path forward for AI applications in the
semiconductor sector.

本文介绍了 Hardware Phi 1.5B，一种专门为半导体行业硬件领域设计的创新大型语言模型，通过使用中等数据集进行预训练，该模型在硬件设计和验证任务中展现出改进的性能，为半导体行业的人工智能应用指明了一个有希望的发展方向。

Hardware Phi-1.5B：一个大型语言模型编码硬件领域特定知识

Hardware Phi-1.5B: A Large Language Model Encodes Hardware Domain  Specific Knowledge

Moore's Law and Dennard Scaling have guided the semiconductor industry for
the past few decades. Recently, both laws have faced validity challenges as
transistor sizes approach the practical limits of physics. We are interested in
testing the validity of these laws and reflect on the reasons responsible. In
this work, we collect data of more than 4000 publicly-available CPU and GPU
products. We find that transistor scaling remains critical in keeping the laws
valid. However, architectural solutions have become increasingly important and
will play a larger role in the future. We observe that GPUs consistently
deliver higher performance than CPUs. GPU performance continues to rise because
of increases in GPU frequency, improvements in the thermal design power (TDP),
and growth in die size. But we also see the ratio of GPU to CPU performance
moving closer to parity, thanks to new SIMD extensions on CPUs and increased
CPU core counts.

本文测试了摩尔定律和丹纳德定律的效力，发现虽然晶体管的缩小仍然是很关键的，但架构方案也变得越来越重要，并且 GPU 在性能上优于 CPU，但得益于 CPU 的新的 SIMD 扩展和核心数量的增加，GPU 和 CPU 在性能上的比例逐渐接近。