The increasing size of Deep Neural Networks (DNNs) poses a pressing need for
model compression, particularly when employed on resource constrained devices.
Concurrently, the susceptibility of DNNs to adversarial attacks presents
another significant hurdle. Despite substantial research on both model
compression and adversarial robustness, their joint examination remains
underexplored. Our study bridges this gap, seeking to understand the effect of
adversarial inputs crafted for base models on their pruned versions. To examine
this relationship, we have developed a comprehensive benchmark across diverse
adversarial attacks and popular DNN models. We uniquely focus on models not
previously exposed to adversarial training and apply pruning schemes optimized
for accuracy and performance. Our findings reveal that while the benefits of
pruning enhanced generalizability, compression, and faster inference times are
preserved, adversarial robustness remains comparable to the base model. This
suggests that model compression while offering its unique advantages, does not
undermine adversarial robustness.

对于基础模型在受挫折性输入下的修剪版本的影响进行了研究，发现在提升普适性、压缩和更快的推断时间方面，模型压缩虽然具有其独特的优势，但不会削弱对抗性鲁棒性。

压缩深度学习模型对抗鲁棒性基准测试

Benchmarking Adversarial Robustness of Compressed Deep Learning Models

Federated learning (FL) is a collaborative learning paradigm for
decentralized private data from mobile terminals (MTs). However, it suffers
from issues in terms of communication, resource of MTs, and privacy. Existing
privacy-preserving FL methods usually adopt the instance-level differential
privacy (DP), which provides a rigorous privacy guarantee but with several
bottlenecks: severe performance degradation, transmission overhead, and
resource constraints of edge devices such as MTs. To overcome these drawbacks,
we propose Fed-LTP, an efficient and privacy-enhanced FL framework with
\underline{\textbf{L}}ottery \underline{\textbf{T}}icket
\underline{\textbf{H}}ypothesis (LTH) and zero-concentrated
D\underline{\textbf{P}} (zCDP). It generates a pruned global model on the
server side and conducts sparse-to-sparse training from scratch with zCDP on
the client side. On the server side, two pruning schemes are proposed: (i) the
weight-based pruning (LTH) determines the pruned global model structure; (ii)
the iterative pruning further shrinks the size of the pruned model's
parameters. Meanwhile, the performance of Fed-LTP is also boosted via model
validation based on the Laplace mechanism. On the client side, we use
sparse-to-sparse training to solve the resource-constraints issue and provide
tighter privacy analysis to reduce the privacy budget. We evaluate the
effectiveness of Fed-LTP on several real-world datasets in both independent and
identically distributed (IID) and non-IID settings. The results clearly confirm
the superiority of Fed-LTP over state-of-the-art (SOTA) methods in
communication, computation, and memory efficiencies while realizing a better
utility-privacy trade-off.

提出了 Fed-LTP，它是一种高效的隐私增强的联邦学习框架，能够通过稀疏到稀疏训练解决资源限制问题，并使用两种修剪方案（即基于权重的修剪和迭代修剪）生成被修剪的全局模型，拥有更好的通信、计算和存储效率，并采用基于拉普拉斯机制的模型验证实现更好的收益隐私权衡。

边缘计算中基于抽奖票剪枝的增强隐私效率联邦学习

Efficient Federated Learning with Enhanced Privacy via Lottery Ticket  Pruning in Edge Computing

Deep neural networks often have millions of parameters. This can hinder their
deployment to low-end devices, not only due to high memory requirements but
also because of increased latency at inference. We propose a novel model
compression method that generates a sparse trained model without additional
overhead: by allowing (i) dynamic allocation of the sparsity pattern and (ii)
incorporating feedback signal to reactivate prematurely pruned weights we
obtain a performant sparse model in one single training pass (retraining is not
needed, but can further improve the performance). We evaluate our method on
CIFAR-10 and ImageNet, and show that the obtained sparse models can reach the
state-of-the-art performance of dense models. Moreover, their performance
surpasses that of models generated by all previously proposed pruning schemes.

提出一种新的模型压缩方法，通过允许稀疏模式的动态分配和合理使用反馈信号使得模型可以在单一训练过程中生成一个高性能的稀疏模型，且其性能超越了现有的所有修剪方案生成的模型，在 CIFAR-10 和 ImageNet 数据集上进行验证。