A counter-intuitive property of convolutional neural networks (CNNs) is their
inherent susceptibility to adversarial examples, which severely hinders the
application of CNNs in security-critical fields. Adversarial examples are
similar to original examples but contain malicious perturbations. Adversarial
training is a simple and effective defense method to improve the robustness of
CNNs to adversarial examples. The mechanisms behind adversarial examples and
adversarial training are worth exploring. Therefore, this work investigates
similarities and differences between normally trained CNNs (NT-CNNs) and
adversarially trained CNNs (AT-CNNs) in information extraction from the mutual
information perspective. We show that 1) whether NT-CNNs or AT-CNNs, for
original and adversarial examples, the trends towards mutual information are
almost similar throughout training; 2) compared with normal training,
adversarial training is more difficult and the amount of information that
AT-CNNs extract from the input is less; 3) the CNNs trained with different
methods have different preferences for certain types of information; NT-CNNs
tend to extract texture-based information from the input, while AT-CNNs prefer
to shape-based information. The reason why adversarial examples mislead CNNs
may be that they contain more texture-based information about other classes.
Furthermore, we also analyze the mutual information estimators used in this
work and find that they outline the geometric properties of the middle layer's
output.

本文研究标准训练 CNN（NT-CNN）和对抗训练 CNN（AT-CNN）在信息提取方面的相似点和不同点，发现对抗训练虽然更加困难，但能使得 CNN 更鲁棒，且能提取形状为主的信息，呈现出互信息随时间递增而递增的趋势。