Recently, deep neural networks have achieved excellent performance on
low-light raw video enhancement. However, they often come with high
computational complexity and large memory costs, which hinder their
applications on resource-limited devices. In this paper, we explore the
feasibility of applying the extremely compact binary neural network (BNN) to
low-light raw video enhancement. Nevertheless, there are two main issues with
binarizing video enhancement models. One is how to fuse the temporal
information to improve low-light denoising without complex modules. The other
is how to narrow the performance gap between binary convolutions with the full
precision ones. To address the first issue, we introduce a spatial-temporal
shift operation, which is easy-to-binarize and effective. The temporal shift
efficiently aggregates the features of neighbor frames and the spatial shift
handles the misalignment caused by the large motion in videos. For the second
issue, we present a distribution-aware binary convolution, which captures the
distribution characteristics of real-valued input and incorporates them into
plain binary convolutions to alleviate the degradation in performance.
Extensive quantitative and qualitative experiments have shown our
high-efficiency binarized low-light raw video enhancement method can attain a
promising performance.

最近，深度神经网络在低光原始视频增强方面取得了出色的性能。然而，它们通常具有高计算复杂性和大内存成本，限制了在资源有限设备上的应用。本文探讨了将极其紧凑的二值神经网络（BNN）应用于低光原始视频增强的可行性，并解决了融合时间信息和二值卷积之间的性能差距问题。经过广泛的定量和定性实验，我们的高效二值化低光原始视频增强方法取得了令人满意的性能。

二值化低光原始视频增强

Binarized Low-light Raw Video Enhancement

We introduce Larq Compute Engine, the world's fastest Binarized Neural
Network (BNN) inference engine, and use this framework to investigate several
important questions about the efficiency of BNNs and to design a new
state-of-the-art BNN architecture. LCE provides highly optimized
implementations of binary operations and accelerates binary convolutions by 8.5
- 18.5x compared to their full-precision counterparts on Pixel 1 phones. LCE's
integration with Larq and a sophisticated MLIR-based converter allow users to
move smoothly from training to deployment. By extending TensorFlow and
TensorFlow Lite, LCE supports models which combine binary and full-precision
layers, and can be easily integrated into existing applications. Using LCE, we
analyze the performance of existing BNN computer vision architectures and
develop QuickNet, a simple, easy-to-reproduce BNN that outperforms existing
binary networks in terms of latency and accuracy on ImageNet. Furthermore, we
investigate the impact of full-precision shortcuts and the relationship between
number of MACs and model latency. We are convinced that empirical performance
should drive BNN architecture design and hope this work will facilitate others
to design, benchmark and deploy binary models.

本研究介绍了 Larq Compute Engine，这是世界上最快的二值（BNN）推理引擎，在此框架下探讨二值网络的效率，设计了一种新的最先进的二值神经网络架构，开发了 QuickNet。经实验证明它在 ImageNet 上的精度和延迟均已超过现有的二值网络。

Larq Compute Engine: 设计、基准测试和部署最先进的二值神经网络

Larq Compute Engine: Design, Benchmark, and Deploy State-of-the-Art  Binarized Neural Networks

Deep neural networks are a biologically-inspired class of algorithms that
have recently demonstrated state-of-the-art accuracies involving large-scale
classification and recognition tasks. Indeed, a major landmark that enables
efficient hardware accelerators for deep networks is the recent advances from
the machine learning community that have demonstrated aggressively scaled deep
binary networks with state-of-the-art accuracies. In this paper, we demonstrate
how deep binary networks can be accelerated in modified von-Neumann machines by
enabling binary convolutions within the SRAM array. In general, binary
convolutions consist of bit-wise XNOR followed by a population-count
(popcount). We present a charge sharing XNOR and popcount operation in 10
transistor SRAM cells. We have employed multiple circuit techniques including
dual-read-worldines (Dual-RWL) along with a dual-stage ADC that overcomes the
inaccuracies of a low precision ADC, to achieve a fairly accurate popcount. In
addition, a key highlight of the present work is the fact that we propose
sectioning of the SRAM array by adding switches onto the read-bitlines, thereby
achieving improved parallelism. This is beneficial for deep networks, where the
kernel size grows and requires to be stored in multiple sub-banks. As such, one
needs to evaluate the partial popcount from multiple sub-banks and sum them up
for achieving the final popcount. For n-sections per sub-array, we can perform
n convolutions within one particular sub-bank, thereby improving overall system
throughput as well as the energy efficiency. Our results at the array level
show that the energy consumption and delay per-operation was 1.914pJ and 45ns,
respectively. Moreover, an energy improvement of 2.5x, and a performance
improvement of 4x was achieved by using the proposed sectioned-SRAM, compared
to a non-sectioned SRAM design.

本文提出了一种在 SRAM 阵列内加速二进制卷积的方法，包括基于电荷共享的 XNOR 和 popcount 运算以及在读位线上添加开关实现 SRAM 阵列分段，采用这种方法可以大大改善整个系统的吞吐量和能耗效率。