Field-Programmable Gate Array (FPGA) accelerators have proven successful in
handling latency- and resource-critical deep neural network (DNN) inference
tasks. Among the most computationally intensive operations in a neural network
(NN) is the dot product between the feature and weight vectors. Thus, some
previous FPGA acceleration works have proposed mapping neurons with quantized
inputs and outputs directly to lookup tables (LUTs) for hardware
implementation. In these works, the boundaries of the neurons coincide with the
boundaries of the LUTs. We propose relaxing these boundaries and mapping entire
sub-networks to a single LUT. As the sub-networks are absorbed within the LUT,
the NN topology and precision within a partition do not affect the size of the
lookup tables generated. Therefore, we utilize fully connected layers with
floating-point precision inside each partition, which benefit from being
universal function approximators, with rigid sparsity and quantization enforced
only between partitions, where the NN topology becomes exposed to the circuit
topology. Although cheap to implement, this approach can lead to very deep NNs,
and so to tackle challenges like vanishing gradients, we also introduce skip
connections inside the partitions. The resulting methodology can be seen as
training DNNs with a specific sparsity pattern that allows them to be mapped to
much shallower circuit-level networks, thereby significantly improving latency.
We validate our proposed method on a known latency-critical task, jet
substructure tagging, and on the classical computer vision task, the digit
classification using MNIST. Our approach allows for greater function
expressivity within the LUTs compared to existing work, leading to lower
latency NNs for the same accuracy.

利用全连接层内的浮点精度作为每个分区内的刚性稀疏性和量化，我们提出了将整个子网络映射到单个查找表（LUT）中的方法，并通过引入跳跃连接来解决梯度消失等挑战，从而显着提高延迟。