Adversarial attacks pose a significant challenge to deploying deep learning
models in safety-critical applications. Maintaining model robustness while
ensuring interpretability is vital for fostering trust and comprehension in
these models. This study investigates the impact of Saliency-guided Training
(SGT) on model robustness, a technique aimed at improving the clarity of
saliency maps to deepen understanding of the model's decision-making process.
Experiments were conducted on standard benchmark datasets using various deep
learning architectures trained with and without SGT. Findings demonstrate that
SGT enhances both model robustness and interpretability. Additionally, we
propose a novel approach combining SGT with standard adversarial training to
achieve even greater robustness while preserving saliency map quality. Our
strategy is grounded in the assumption that preserving salient features crucial
for correctly classifying adversarial examples enhances model robustness, while
masking non-relevant features improves interpretability. Our technique yields
significant gains, achieving a 35\% and 20\% improvement in robustness against
PGD attack with noise magnitudes of $0.2$ and $0.02$ for the MNIST and CIFAR-10
datasets, respectively, while producing high-quality saliency maps.

通过使用基于显著性引导的训练（SGT）技术，研究表明它对深度学习模型的鲁棒性和可解释性有显著提升，并提出了一种将 SGT 与标准对抗训练相结合的方法，在确保显著性图质量的同时，实现更大程度的鲁棒性。该方法取得了显著进展，在 MNIST 和 CIFAR-10 数据集上，对于 PGD 攻击的噪声幅度分别为 0.2 和 0.02 时，鲁棒性提高了 35％和 20％，同时生成高质量的显著性图。

深度神经网络中解释性与鲁棒性的相互作用探索：一种基于显著性引导的方法

Exploring the Interplay of Interpretability and Robustness in Deep  Neural Networks: A Saliency-guided Approach

Deep Neural Networks are powerful tools for understanding complex patterns
and making decisions. However, their black-box nature impedes a complete
understanding of their inner workings. Saliency-Guided Training (SGT) methods
try to highlight the prominent features in the model's training based on the
output to alleviate this problem. These methods use back-propagation and
modified gradients to guide the model toward the most relevant features while
keeping the impact on the prediction accuracy negligible. SGT makes the model's
final result more interpretable by masking input partially. In this way,
considering the model's output, we can infer how each segment of the input
affects the output. In the particular case of image as the input, masking is
applied to the input pixels. However, the masking strategy and number of pixels
which we mask, are considered as a hyperparameter. Appropriate setting of
masking strategy can directly affect the model's training. In this paper, we
focus on this issue and present our contribution. We propose a novel method to
determine the optimal number of masked images based on input, accuracy, and
model loss during the training. The strategy prevents information loss which
leads to better accuracy values. Also, by integrating the model's performance
in the strategy formula, we show that our model represents the salient features
more meaningful. Our experimental results demonstrate a substantial improvement
in both model accuracy and the prominence of saliency, thereby affirming the
effectiveness of our proposed solution.

通过提出一种基于输入、准确性和模型损失的遮蔽图像数目的最佳确定方法，本文旨在解决遮蔽策略的问题，以提高模型的准确性和突出性，从而证明所提出的解决方案的有效性。

SMOOT：基于显著性引导的掩模优化在线训练

SMOOT: Saliency Guided Mask Optimized Online Training

Deep Neural Networks are powerful tools to understand complex patterns and
making decisions. However, their black-box nature impedes a complete
understanding of their inner workings. While online saliency-guided training
methods try to highlight the prominent features in the model's output to
alleviate this problem, it is still ambiguous if the visually explainable
features align with robustness of the model against adversarial examples. In
this paper, we investigate the saliency trained model's vulnerability to
adversarial examples methods. Models are trained using an online
saliency-guided training method and evaluated against popular algorithms of
adversarial examples. We quantify the robustness and conclude that despite the
well-explained visualizations in the model's output, the salient models suffer
from the lower performance against adversarial examples attacks.

本文研究深度神经网络的鲁棒性问题，特别关注对抗样本的攻击。通过在线显著特征训练，得到视觉可解释性的特征，但实验结果显示，这些特征并不能提升模型对抗样本攻击的性能。