Deep Learning has established itself to be a common occurrence in the business lexicon. The unprecedented success of deep learning in recent years can be attributed to: abundance of data, availability of gargantuan compute capabilities offered by GPUs, and adoption of open-source philosophy by the researchers and industry. Deep neural networks can be decomposed into a series of different operators. MIOpen, AMD's open-source deep learning primitives library for GPUs, provides highly optimized implementations of such operators, shielding researchers from internal implementation details and hence, accelerating the time to discovery. This paper introduces MIOpen and provides details about the internal workings of the library and supported features. MIOpen innovates on several fronts, such as implementing fusion to optimize for memory bandwidth and GPU launch overheads, providing an auto-tuning infrastructure to overcome the large design space of problem configurations, and implementing different algorithms to optimize convolutions for different filter and input sizes. MIOpen is one of the first libraries to publicly support the bfloat16 data-type for convolutions, allowing efficient training at lower precision without the loss of accuracy.

本文介绍了AMD公司开源的深度学习基元库MIOpen，讨论了其内部工作方式和支持特性，提出了该库的一些创新变化，如实现内存带宽和GPU启动开销的融合优化，提供自动调整基础架构以克服问题配置的设计空间，以及实现了不同的算法来优化过滤器和不同输入尺寸的卷积，是第一个公开支持bfloat16数据类型进行卷积的库，允许在低精度下进行高效的训练，而不会丢失精度。

MIOpen: 一个开源深度学习基础库