Resistive Random Access Memory (ReRAM) has emerged as a promising platform
for deep neural networks (DNNs) due to its support for parallel in-situ
matrix-vector multiplication. However, hardware failures, such as
stuck-at-fault defects, can result in significant prediction errors during
model inference. While additional crossbars can be used to address these
failures, they come with storage overhead and are not efficient in terms of
space, energy, and cost. In this paper, we propose a fault protection mechanism
that incurs zero space cost. Our approach includes: 1) differentiable structure
pruning of rows and columns to reduce model redundancy, 2) weight duplication
and voting for robust output, and 3) embedding duplicated most significant bits
(MSBs) into the model weight. We evaluate our method on nine tasks of the GLUE
benchmark with the BERT model, and experimental results prove its
effectiveness.

本文提出了一种零空间成本的容错机制，通过可微结构修剪、权重复制和投票以及嵌入最显著位（MSBs）到模型权重中，有效减少了硬件故障对模型预测的影响，并通过在 GLUE 基准的九个任务上使用 BERT 模型进行实验验证了该方法的有效性。

基于 ReRAM 的 Transformer 语言模型的零空间成本容错技术

Zero-Space Cost Fault Tolerance for Transformer-based Language Models on  ReRAM

Convolution neural networks (CNNs) have achieved remarkable success, but
typically accompany high computation cost and numerous redundant weight
parameters. To reduce the FLOPs, structure pruning is a popular approach to
remove the entire hidden structures via introducing coarse-grained sparsity.
Meanwhile, plentiful pruning works leverage fine-grained sparsity instead
(sparsity are randomly distributed), whereas their sparse models lack special
designed computing library for potential speedup. In this technical report, we
study and present an efficient convolution neural network inference system to
accelerate its forward pass by utilizing the fine-grained sparsity of
compressed CNNs. Our developed FSCNN is established based on a set of
specialized designed sparse data structures, operators and associated
algorithms. Experimentally, we validate that FSCNN outperforms standard deep
learning library PyTorch on popular CNN architectures such as VGG16 if
sufficiently high sparsity exhibits. However, due to the contiguity issue of
sparse operators, FSCNN is typically not comparable with highly optimized dense
operator. Therefore, coarse-grained (structured) sparsity is our recommendation
for generic model compression.

本文主要介绍了利用大规模稀疏结构提高卷积神经网络推理效率的方法， 开发了一种名为 FSCNN 的稀疏卷积神经网络推理系统，能够在一定的稀疏程度下优于标准深度学习库 PyTorch，但遇到高优化密集运算时推理效率有一定局限性，因此推荐采用粗粒度稀疏度进行通用模型压缩。