Binary Neural Networks (BNNs) can significantly accelerate the inference time
of a neural network by replacing its expensive floating-point arithmetic with
bitwise operations. Most existing solutions, however, do not fully optimize
data flow through the BNN layers, and intermediate conversions from 1 to 16/32
bits often further hinder efficiency. We propose a novel training scheme that
can increase data flow and parallelism in the BNN pipeline; specifically, we
introduce a clipping block that decreases the data-width from 32 bits to 8.
Furthermore, we reduce the internal accumulator size of a binary layer, usually
kept using 32-bit to prevent data overflow without losing accuracy.
Additionally, we provide an optimization of the Batch Normalization layer that
both reduces latency and simplifies deployment. Finally, we present an
optimized implementation of the Binary Direct Convolution for ARM instruction
sets. Our experiments show a consistent improvement of the inference speed (up
to 1.91 and 2.73x compared to two state-of-the-art BNNs frameworks) with no
drop in accuracy for at least one full-precision model.

通过引入剪裁块，减少二元神经网络层的数据宽度和内部累加器大小，同时优化批标准化层的实现和 ARM 指令集的二值化卷积，提高了推理速度和准确度。