StyleGAN has shown remarkable performance in unconditional image generation.
However, its high computational cost poses a significant challenge for
practical applications. Although recent efforts have been made to compress
StyleGAN while preserving its performance, existing compressed models still lag
behind the original model, particularly in terms of sample diversity. To
overcome this, we propose a novel channel pruning method that leverages varying
sensitivities of channels to latent vectors, which is a key factor in sample
diversity. Specifically, by assessing channel importance based on their
sensitivities to latent vector perturbations, our method enhances the diversity
of samples in the compressed model. Since our method solely focuses on the
channel pruning stage, it has complementary benefits with prior training
schemes without additional training cost. Extensive experiments demonstrate
that our method significantly enhances sample diversity across various
datasets. Moreover, in terms of FID scores, our method not only surpasses
state-of-the-art by a large margin but also achieves comparable scores with
only half training iterations.

我们提出了一种新颖的通道修剪方法，通过评估通道对潜在向量扰动的敏感性，增强了压缩模型中样本的多样性。我们的方法显著提升了各种数据集中样本的多样性，并且在 FID 得分方面，不仅大幅超过了最先进的方法，还在只有一半训练迭代次数的情况下达到了可比较的得分。

基于多样性感知的 StyleGAN 压缩通道剪枝

Diversity-aware Channel Pruning for StyleGAN Compression

Unsupervised deep learning techniques are widely used to identify anomalous
behaviour. The performance of such methods is a product of the amount of
training data and the model size. However, the size is often a limiting factor
for the deployment on resource-constrained devices. We present a novel
procedure based on knowledge distillation for compressing an unsupervised
anomaly detection model into a supervised deployable one and we suggest a set
of techniques to improve the detection sensitivity. Compressed models perform
comparably to their larger counterparts while significantly reducing the size
and memory footprint.

无监督深度学习技术广泛用于识别异常行为。该研究介绍了一种基于知识蒸馏的新型过程，用于将无监督异常检测模型压缩成监督可部署模型，并提出了一套提高检测敏感性的技术集合。压缩模型在显著减小大小和内存占用的同时，性能与较大模型相当。

知识蒸馏用于异常检测

Knowledge Distillation for Anomaly Detection

Designing deep learning-based solutions is becoming a race for training
deeper models with a greater number of layers. While a large-size deeper model
could provide competitive accuracy, it creates a lot of logistical challenges
and unreasonable resource requirements during development and deployment. This
has been one of the key reasons for deep learning models not being excessively
used in various production environments, especially in edge devices. There is
an immediate requirement for optimizing and compressing these deep learning
models, to enable on-device intelligence. In this research, we introduce a
black-box framework, Deeplite Neutrino for production-ready optimization of
deep learning models. The framework provides an easy mechanism for the
end-users to provide constraints such as a tolerable drop in accuracy or target
size of the optimized models, to guide the whole optimization process. The
framework is easy to include in an existing production pipeline and is
available as a Python Package, supporting PyTorch and Tensorflow libraries. The
optimization performance of the framework is shown across multiple benchmark
datasets and popular deep learning models. Further, the framework is currently
used in production and the results and testimonials from several clients are
summarized.

该研究介绍了一种基于深度学习的黑盒优化框架，Deeplite Neutrino，旨在优化深度学习模型的大小，从而在边缘设备上实现设备上智能应用。该框架易于将其纳入现有生产流程并支持 PyTorch 和 Tensorflow 库。同时，该框架的性能经过了多个基准数据集和流行的深度学习模型的优化，且在多个客户端获得了成果和证明。