Despite their unprecedented success, DNNs are notoriously fragile to small
shifts in data distribution, demanding effective testing techniques that can
assess their dependability. Despite recent advances in DNN testing, there is a
lack of systematic testing approaches that assess the DNN's capability to
generalise and operate comparably beyond data in their training distribution.
We address this gap with DeepKnowledge, a systematic testing methodology for
DNN-based systems founded on the theory of knowledge generalisation, which aims
to enhance DNN robustness and reduce the residual risk of 'black box' models.
Conforming to this theory, DeepKnowledge posits that core computational DNN
units, termed Transfer Knowledge neurons, can generalise under domain shift.
DeepKnowledge provides an objective confidence measurement on testing
activities of DNN given data distribution shifts and uses this information to
instrument a generalisation-informed test adequacy criterion to check the
transfer knowledge capacity of a test set. Our empirical evaluation of several
DNNs, across multiple datasets and state-of-the-art adversarial generation
techniques demonstrates the usefulness and effectiveness of DeepKnowledge and
its ability to support the engineering of more dependable DNNs. We report
improvements of up to 10 percentage points over state-of-the-art coverage
criteria for detecting adversarial attacks on several benchmarks, including
MNIST, SVHN, and CIFAR.

通过 DeepKnowledge 系统测试方法，可以评估 DNN 的可靠性和泛化能力，并通过提供测量信息和检查测试集的传输知识容量来改善 DNN 的稳健性。

深度知识：基于泛化的深度学习测试

DeepKnowledge: Generalisation-Driven Deep Learning Testing

Compute-in-Memory (CiM) utilizing non-volatile memory (NVM) devices presents
a highly promising and efficient approach for accelerating deep neural networks
(DNNs). By concurrently storing network weights and performing matrix
operations within the same crossbar structure, CiM accelerators offer DNN
inference acceleration with minimal area requirements and exceptional energy
efficiency. However, the stochasticity and intrinsic variations of NVM devices
often lead to performance degradation, such as reduced classification accuracy,
compared to expected outcomes. Although several methods have been proposed to
mitigate device variation and enhance robustness, most of them rely on overall
modulation and lack constraints on the training process. Drawing inspiration
from the negative feedback mechanism, we introduce a novel training approach
that uses a multi-exit mechanism as negative feedback to enhance the
performance of DNN models in the presence of device variation. Our negative
feedback training method surpasses state-of-the-art techniques by achieving an
impressive improvement of up to 12.49% in addressing DNN robustness against
device variation.

本研究提出一种新的深度神经网络 DNN 训练方法，利用负反馈机制作为多出口机制来提高神经网络模型对设备变异的鲁棒性，在处理 Compute-in-Memory，非易失性内存，矩阵操作等方面都有很好的性能和效率。

负反馈训练：提高 NVCiM DNN 加速器稳健性的新概念

Negative Feedback Training: A Novel Concept to Improve Robustness of  NVCiM DNN Accelerators

Neural architecture search (NAS) has emerged as one successful technique to
find robust deep neural network (DNN) architectures. However, most existing
robustness evaluations in NAS only consider $l_{\infty}$ norm-based adversarial
noises. In order to improve the robustness of DNN models against multiple types
of noises, it is necessary to consider a comprehensive evaluation in NAS for
robust architectures. But with the increasing number of types of robustness
evaluations, it also becomes more time-consuming to find comprehensively robust
architectures. To alleviate this problem, we propose a novel efficient search
of comprehensively robust neural architectures via multi-fidelity evaluation
(ES-CRNA-ME). Specifically, we first search for comprehensively robust
architectures under multiple types of evaluations using the
weight-sharing-based NAS method, including different $l_{p}$ norm attacks,
semantic adversarial attacks, and composite adversarial attacks. In addition,
we reduce the number of robustness evaluations by the correlation analysis,
which can incorporate similar evaluations and decrease the evaluation cost.
Finally, we propose a multi-fidelity online surrogate during optimization to
further decrease the search cost. On the basis of the surrogate constructed by
low-fidelity data, the online high-fidelity data is utilized to finetune the
surrogate. Experiments on CIFAR10 and CIFAR100 datasets show the effectiveness
of our proposed method.

通过多重鲁棒性评估的权值共享 NAS 方法，提出了一种高效的多层次鲁棒神经结构搜索方法，可以在多种攻击类型下搜索到具有高鲁棒性的神经结构。