We introduce a new algorithm called the Free-pipeline Fast Inner Product
(FFIP) and its hardware architecture that improve an under-explored fast
inner-product algorithm (FIP) proposed by Winograd in 1968. Unlike the
unrelated Winograd minimal filtering algorithms for convolutional layers, FIP
is applicable to all machine learning (ML) model layers that can mainly
decompose to matrix multiplication, including fully-connected, convolutional,
recurrent, and attention/transformer layers. We implement FIP for the first
time in an ML accelerator then present our FFIP algorithm and generalized
architecture which inherently improve FIP's clock frequency and, as a
consequence, throughput for a similar hardware cost. Finally, we contribute
ML-specific optimizations for the FIP and FFIP algorithms and architectures. We
show that FFIP can be seamlessly incorporated into traditional fixed-point
systolic array ML accelerators to achieve the same throughput with half the
number of multiply-accumulate (MAC) units, or it can double the maximum
systolic array size that can fit onto devices with a fixed hardware budget. Our
FFIP implementation for non-sparse ML models with 8 to 16-bit fixed-point
inputs achieves higher throughput and compute efficiency than the best-in-class
prior solutions on the same type of compute platform.

提出了一种名为自由流水线快速内积（FFIP）的新算法及其硬件架构，该算法改进了 Winograd 在 1968 年提出的一种未被充分探索的快速内积算法（FIP）。与 Winograd 最小滤波算法不同，FIP 适用于所有可分解为矩阵乘法的机器学习模型层，包括全连接、卷积、循环和注意力 / 变换层。我们首次在机器学习加速器中实现了 FIP，提出了 FFIP 算法和通用架构，从而从本质上提高了 FIP 的时钟频率和吞吐量。此外，我们对 FIP 和 FFIP 算法及架构进行了机器学习特定的优化。我们发现，在相同类型的计算平台上，FFIP 对于非稀疏机器学习模型的 8 到 16 位定点输入实现了比同类先前解决方案更高的吞吐量和计算效率。