Vision Transformers (ViTs) have demonstrated remarkable performance in image classification tasks, particularly when equipped with local information via region attention or convolutions. While such architectures improve the feature aggregation from different granularities, they often fail to contribute to the robustness of the networks. Neural Cellular Automata (NCA) enables the modeling of global cell representations through local interactions, with its training strategies and architecture design conferring strong generalization ability and robustness against noisy inputs. In this paper, we propose Adaptor Neural Cellular Automata (AdaNCA) for Vision Transformer that uses NCA as plug-in-play adaptors between ViT layers, enhancing ViT's performance and robustness against adversarial samples as well as out-of-distribution inputs. To overcome the large computational overhead of standard NCAs, we propose Dynamic Interaction for more efficient interaction learning. Furthermore, we develop an algorithm for identifying the most effective insertion points for AdaNCA based on our analysis of AdaNCA placement and robustness improvement. With less than a 3% increase in parameters, AdaNCA contributes to more than 10% absolute improvement in accuracy under adversarial attacks on the ImageNet1K benchmark. Moreover, we demonstrate with extensive evaluations across 8 robustness benchmarks and 4 ViT architectures that AdaNCA, as a plug-in-play module, consistently improves the robustness of ViTs.

本文提出了一种称为AdaNCA的适配器神经元细胞自动机（Adaptor Neural Cellular Automata，AdaNCA）的方法，用于Vision Transformer，通过在Vision Transformer层之间插入NCA来增强ViT的性能和抵抗对抗攻击以及在分布之外的输入时的鲁棒性。在ImageNet1K基准测试中，AdaNCA仅增加不到3%的参数，并使准确度在对抗攻击下提高了超过10%的绝对值。此外，通过对8个鲁棒性基准测试和4个ViT架构进行广泛评估，证明了AdaNCA作为插件模块能够一致提高ViTs的鲁棒性。

AdaNCA:神经元元胞自动机作为更为鲁棒的视觉变换器的适配器