Neural Architecture Search (NAS) paves the way for the automatic definition
of Neural Network (NN) architectures, attracting increasing research attention
and offering solutions in various scenarios. This study introduces a novel NAS
solution, called Flat Neural Architecture Search (FlatNAS), which explores the
interplay between a novel figure of merit based on robustness to weight
perturbations and single NN optimization with Sharpness-Aware Minimization
(SAM). FlatNAS is the first work in the literature to systematically explore
flat regions in the loss landscape of NNs in a NAS procedure, while jointly
optimizing their performance on in-distribution data, their out-of-distribution
(OOD) robustness, and constraining the number of parameters in their
architecture. Differently from current studies primarily concentrating on OOD
algorithms, FlatNAS successfully evaluates the impact of NN architectures on
OOD robustness, a crucial aspect in real-world applications of machine and deep
learning. FlatNAS achieves a good trade-off between performance, OOD
generalization, and the number of parameters, by using only in-distribution
data in the NAS exploration. The OOD robustness of the NAS-designed models is
evaluated by focusing on robustness to input data corruptions, using popular
benchmark datasets in the literature.

该研究引入了一种名为 Flat Neural Architecture Search (FlatNAS) 的新型 NAS 解决方案，其探索了基于对权重扰动的鲁棒性和使用 Sharpness-Aware Minimization (SAM) 进行单个 NN 优化之间的相互作用。FlatNAS 是文献中首个系统性地探索 NN 损失景观中的平坦区域的 NAS 方法，并同时优化其在分布数据、OOD 鲁棒性和体系结构参数数量上的性能。与当前主要关注 OOD 算法的研究不同，FlatNAS 成功地评估了 NN 体系结构对 OOD 鲁棒性的影响，这在机器和深度学习的真实应用中是一个关键因素。FlatNAS 通过仅使用分布数据在 NAS 探索中达到了性能、OOD 泛化和参数数量之间的良好平衡。通过在文献中使用流行的基准数据集，评估了 NAS 设计的模型对输入数据破坏的鲁棒性。