We propose a Digit-Serial Left-tO-righT (DSLOT) arithmetic based processing
technique called DSLOT-NN with aim to accelerate inference of the convolution
operation in the deep neural networks (DNNs). The proposed work has the ability
to assess and terminate the ineffective convolutions which results in massive
power and energy savings. The processing engine is comprised of low-latency
most-significant-digit-first (MSDF) (also called online) multipliers and adders
that processes data from left-to-right, allowing the execution of subsequent
operations in digit-pipelined manner. Use of online operators eliminates the
need for the development of complex mechanism of identifying the negative
activation, as the output with highest weight value is generated first, and the
sign of the result can be identified as soon as first non-zero digit is
generated. The precision of the online operators can be tuned at run-time,
making them extremely useful in situations where accuracy can be compromised
for power and energy savings. The proposed design has been implemented on
Xilinx Virtex-7 FPGA and is compared with state-of-the-art Stripes on various
performance metrics. The results show the proposed design presents power
savings, has shorter cycle time, and approximately 50% higher OPS per watt.

我们提出了一种基于 DSLOT-NN 的 Digit-Serial Left-tO-righT（DSLOT）算术处理技术，旨在加速深度神经网络（DNNs）中卷积操作的推断。该提出的方法能够评估和停止无效的卷积操作，从而实现大规模的功率和能量节省。

DSLOT-NN：数字串行从左到右神经网络加速器

DSLOT-NN: Digit-Serial Left-to-Right Neural Network Accelerator

Hardware-based acceleration is an extensive attempt to facilitate many
computationally-intensive mathematics operations. This paper proposes an
FPGA-based architecture to accelerate the convolution operation - a complex and
expensive computing step that appears in many Convolutional Neural Network
models. We target the design to the standard convolution operation, intending
to launch the product as an edge-AI solution. The project's purpose is to
produce an FPGA IP core that can process a convolutional layer at a time.
System developers can deploy the IP core with various FPGA families by using
Verilog HDL as the primary design language for the architecture. The
experimental results show that our single computing core synthesized on a
simple edge computing FPGA board can offer 0.224 GOPS. When the board is fully
utilized, 4.48 GOPS can be achieved.

本文介绍了一种基于硬件加速的卷积操作 FPGA 架构，旨在实现单层卷积的处理，以推出一种边缘 AI 解决方案 IP 核，实验结果表明它的峰值运算性能可达 4.48 GOPS。

基于 FPGA 的卷积运算加速解决方案

An FPGA-based Solution for Convolution Operation Acceleration

Several recent studies have demonstrated that attention-based networks, such
as Vision Transformer (ViT), can outperform Convolutional Neural Networks
(CNNs) on several computer vision tasks without using convolutional layers.
This naturally leads to the following questions: Can a self-attention layer of
ViT express any convolution operation? In this work, we prove that a single ViT
layer with image patches as the input can perform any convolution operation
constructively, where the multi-head attention mechanism and the relative
positional encoding play essential roles. We further provide a lower bound on
the number of heads for Vision Transformers to express CNNs. Corresponding with
our analysis, experimental results show that the construction in our proof can
help inject convolutional bias into Transformers and significantly improve the
performance of ViT in low data regimes.

本文介绍视觉 Transformer (ViT) 在使用自注意力机制的基础上，探究其能否表达卷积操作，并证明使用输入图像块的单个 ViT 层可以构建任何卷积操作，其中多头注意机制和相对位置编码起着关键作用。作者还提供了 Vision Transformer 表达 CNN 所需头数的下限，该证明的构建可以帮助将卷积偏差注入 Transformer，并在低数据环境下显著提高 ViT 的性能。