Deep neural networks (DNN) have shown superior performance in a variety of
tasks. As they rapidly evolve, their escalating computation and memory demands
make it challenging to deploy them on resource-constrained edge devices. Though
extensive efficient accelerator designs, from traditional electronics to
emerging photonics, have been successfully demonstrated, they are still
bottlenecked by expensive memory accesses due to tremendous gaps between the
bandwidth/power/latency of electrical memory and computing cores. Previous
solutions fail to fully-leverage the ultra-fast computational speed of emerging
DNN accelerators to break through the critical memory bound. In this work, we
propose a general and unified framework to trade expensive memory transactions
with ultra-fast on-chip computations, directly translating to performance
improvement. We are the first to jointly explore the intrinsic correlations and
bit-level redundancy within DNN kernels and propose a multi-level in situ
generation mechanism with mixed-precision bases to achieve on-the-fly recovery
of high-resolution parameters with minimum hardware overhead. Extensive
experiments demonstrate that our proposed joint method can boost the memory
efficiency by 10-20x with comparable accuracy over four state-of-the-art
designs, when benchmarked on ResNet-18/DenseNet-121/MobileNetV2/V3 with various
tasks.

本研究提出了一种通用和统一的框架，通过多级现场生成机制和混合精度基准，实现了高分辨率参数的即时恢复，从而以最小的硬件开销直接将昂贵的内存交易转换为超快的芯片内计算，提高了内存效率 10-20 倍。