Recent advances in adversarial attacks uncover the intrinsic vulnerability of
modern deep neural networks. Since then, extensive efforts have been devoted to
enhancing the robustness of deep networks via specialized learning algorithms
and loss functions. In this work, we take an architectural perspective and
investigate the patterns of network architectures that are resilient to
adversarial attacks. To obtain the large number of networks needed for this
study, we adopt one-shot neural architecture search, training a large network
for once and then finetuning the sub-networks sampled therefrom. The sampled
architectures together with the accuracies they achieve provide a rich basis
for our study. Our "robust architecture Odyssey" reveals several valuable
observations: 1) densely connected patterns result in improved robustness; 2)
under computational budget, adding convolution operations to direct connection
edge is effective; 3) flow of solution procedure (FSP) matrix is a good
indicator of network robustness. Based on these observations, we discover a
family of robust architectures (RobNets). On various datasets, including CIFAR,
SVHN, Tiny-ImageNet, and ImageNet, RobNets exhibit superior robustness
performance to other widely used architectures. Notably, RobNets substantially
improve the robust accuracy (~5% absolute gains) under both white-box and
black-box attacks, even with fewer parameter numbers. Code is available at
this https URL

通过一次性神经网络结构搜索，研究了对对抗攻击弹性更强的网络结构模式，并基于这些模式提出了一族更具弹性的网络结构 RobNets。这些结构在多个数据集上表现出更优异的弹性表现，即使具有更少的参数数量，在白盒和黑盒攻击下都表现出 5% 的绝对收益。