The state-of-art DNN structures involve intensive computation and high memory
storage. To mitigate the challenges, the memristor crossbar array has emerged
as an intrinsically suitable matrix computation and low-power acceleration
framework for DNN applications. However, the high accuracy solution for extreme
model compression on memristor crossbar array architecture is still waiting for
unraveling. In this paper, we propose a memristor-based DNN framework which
combines both structured weight pruning and quantization by incorporating
alternating direction method of multipliers (ADMM) algorithm for better pruning
and quantization performance. We also discover the non-optimality of the ADMM
solution in weight pruning and the unused data path in a structured pruned
model. Motivated by these discoveries, we design a software-hardware
co-optimization framework which contains the first proposed Network
Purification and Unused Path Removal algorithms targeting on post-processing a
structured pruned model after ADMM steps. By taking memristor hardware
constraints into our whole framework, we achieve extreme high compression ratio
on the state-of-art neural network structures with minimum accuracy loss. For
quantizing structured pruned model, our framework achieves nearly no accuracy
loss after quantizing weights to 8-bit memristor weight representation. We
share our models at anonymous link this https URL

本文提出了一种基于 memristor 的 DNN 框架，该框架将结构化权重修剪和量化相结合，利用交替方向乘子法（ADMM）算法进行更好的修剪和量化性能，并设计了一种软硬件优化框架，实现了高压缩比和几乎无精度损失的量化。