The development of model compression is continuously motivated by the
evolution of various neural network accelerators with ASIC or FPGA. On the
algorithm side, the ultimate goal of quantization or pruning is accelerating
the expensive DNN computations on low-power hardware. However, such a
"design-and-deploy" workflow faces under-explored challenges in the current
hardware-algorithm co-design community. First, although the state-of-the-art
quantization algorithm can achieve low precision with negligible degradation of
accuracy, the latest deep learning framework (e.g., PyTorch) can only support
non-customizable 8-bit precision, data format, and parameter extraction.
Secondly, the objective of quantization is to enable the computation with
low-precision data. However, the current SoTA algorithm treats the quantized
integer as an intermediate result, while the final output of the quantizer is
the "discretized" floating-point values, ignoring the practical needs and
adding additional workload to hardware designers for integer parameter
extraction and layer fusion. Finally, the compression toolkits designed by the
industry are constrained to their in-house product or a handful of algorithms.
The limited degree of freedom in the current toolkit and the under-explored
customization hinder the prototype ASIC or FPGA-based accelerator design. To
resolve these challenges, we propose Torch2Chip, an open-sourced, fully
customizable, and high-performance toolkit that supports user-designed
compression followed by automatic model fusion and parameter extraction.
Torch2Chip incorporates the hierarchical design workflow, and the
user-customized compression algorithm will be directly packed into the
deployment-ready format for prototype chip verification with either CNN or
vision transformer (ViT). The code is available at
this https URL

通过自定义的模型压缩算法实现量化，从而加速在低功耗硬件上进行昂贵的深度神经网络计算的开源工具 Torch2Chip。

Torch2Chip: 用于原型硬件加速器设计的端到端可定制的深度神经网络压缩和部署工具包

Torch2Chip: An End-to-end Customizable Deep Neural Network Compression  and Deployment Toolkit for Prototype Hardware Accelerator Design

Pruning methods have shown to be effective at reducing the size of deep
neural networks while keeping accuracy almost intact. Among the most effective
methods are those that prune a network while training it with a sparsity prior
loss and learnable dropout parameters. A shortcoming of these approaches
however is that neither the size nor the inference speed of the pruned network
can be controlled directly; yet this is a key feature for targeting deployment
of CNNs on low-power hardware. To overcome this, we introduce a budgeted
regularized pruning framework for deep CNNs. Our approach naturally fits into
traditional neural network training as it consists of a learnable masking
layer, a novel budget-aware objective function, and the use of knowledge
distillation. We also provide insights on how to prune a residual network and
how this can lead to new architectures. Experimental results reveal that CNNs
pruned with our method are more accurate and less compute-hungry than
state-of-the-art methods. Also, our approach is more effective at preventing
accuracy collapse in case of severe pruning; this allows us to attain pruning
factors up to 16x without significant accuracy drop.

本研究提出了一种基于预算限制的深度卷积神经网络修剪框架，并利用学习掩码层、创新的预算目标函数以及知识蒸馏等技术进行修剪，实验结果表明，使用我们的方法修剪后的 CNNs 比现有方法更准确、计算更少，同时即使极度剪枝也不影响分类准确度，具有很大的实用价值。