Unsupervised anomaly detection encompasses diverse applications in industrial
settings where a high-throughput and precision is imperative. Early works were
centered around one-class-one-model paradigm, which poses significant
challenges in large-scale production environments. Knowledge-distillation based
multi-class anomaly detection promises a low latency with a reasonably good
performance but with a significant drop as compared to one-class version. We
propose a DCAM (Distributed Convolutional Attention Module) which improves the
distillation process between teacher and student networks when there is a high
variance among multiple classes or objects. Integrated multi-scale feature
matching strategy to utilise a mixture of multi-level knowledge from the
feature pyramid of the two networks, intuitively helping in detecting anomalies
of varying sizes which is also an inherent problem in the multi-class scenario.
Briefly, our DCAM module consists of Convolutional Attention blocks distributed
across the feature maps of the student network, which essentially learns to
masks the irrelevant information during student learning alleviating the
"cross-class interference" problem. This process is accompanied by minimizing
the relative entropy using KL-Divergence in Spatial dimension and a
Channel-wise Cosine Similarity between the same feature maps of teacher and
student. The losses enables to achieve scale-invariance and capture non-linear
relationships. We also highlight that the DCAM module would only be used during
training and not during inference as we only need the learned feature maps and
losses for anomaly scoring and hence, gaining a performance gain of 3.92% than
the multi-class baseline with a preserved latency.

我们提出了一种基于 DCAM（分布式卷积注意模块）的知识蒸馏方法，可以改善教师和学生网络之间的蒸馏过程，减少多类别或多对象情况下的交叉类干扰问题，并通过最小化空间维度上的相对熵和教师和学生的相同特征图之间的通道相关余弦相似性来实现尺度不变性和捕捉非线性关系。

参与、提炼、检测：注意力感知的熵提炼用于异常检测

Attend, Distill, Detect: Attention-aware Entropy Distillation for  Anomaly Detection

Self-supervised learned models have been found to be very effective for
certain speech tasks such as automatic speech recognition, speaker
identification, keyword spotting and others. While the features are undeniably
useful in speech recognition and associated tasks, their utility in speech
enhancement systems is yet to be firmly established, and perhaps not properly
understood. In this paper, we investigate the uses of SSL representations for
single-channel speech enhancement in challenging conditions and find that they
add very little value for the enhancement task. Our constraints are designed
around on-device real-time speech enhancement -- model is causal, the compute
footprint is small. Additionally, we focus on low SNR conditions where such
models struggle to provide good enhancement. In order to systematically examine
how SSL representations impact performance of such enhancement models, we
propose a variety of techniques to utilize these embeddings which include
different forms of knowledge-distillation and pre-training.

通过对自监督学习表示进行研究，我们发现它们在挑战性条件下的单通道语音增强任务中几乎没有增加价值。为了系统地评估 SSL 表示对这些增强模型性能的影响，我们提出了多种利用这些嵌入的技术，包括不同形式的知识蒸馏和预训练。