Prior neural architecture search (NAS) for adversarial robustness works have
discovered that a lightweight and adversarially robust neural network
architecture could exist in a non-robust large teacher network, generally
disclosed by heuristic rules through statistical analysis and neural
architecture search, generally disclosed by heuristic rules from neural
architecture search. However, heuristic methods cannot uniformly handle
different adversarial attacks and "teacher" network capacity. To solve this
challenge, we propose a Reinforced Compressive Neural Architecture Search
(RC-NAS) for Versatile Adversarial Robustness. Specifically, we define task
settings that compose datasets, adversarial attacks, and teacher network
information. Given diverse tasks, we conduct a novel dual-level training
paradigm that consists of a meta-training and a fine-tuning phase to
effectively expose the RL agent to diverse attack scenarios (in meta-training),
and making it adapt quickly to locate a sub-network (in fine-tuning) for any
previously unseen scenarios. Experiments show that our framework could achieve
adaptive compression towards different initial teacher networks, datasets, and
adversarial attacks, resulting in more lightweight and adversarially robust
architectures.

通过双层训练范式和适应多样攻击情境的框架，我们提出了一种适应性压缩神经构架搜索方法，使得神经网络具备更轻量级和更具鲁棒性。