Recent advancements in Zero-shot Neural Architecture Search (NAS) highlight
the efficacy of zero-cost proxies in various NAS benchmarks. Several studies
propose the automated design of zero-cost proxies to achieve SOTA performance
but require tedious searching progress. Furthermore, we identify a critical
issue with current zero-cost proxies: they aggregate node-wise zero-cost
statistics without considering the fact that not all nodes in a neural network
equally impact performance estimation. Our observations reveal that node-wise
zero-cost statistics significantly vary in their contributions to performance,
with each node exhibiting a degree of uncertainty. Based on this insight, we
introduce a novel method called Parametric Zero-Cost Proxies (ParZC) framework
to enhance the adaptability of zero-cost proxies through parameterization. To
address the node indiscrimination, we propose a Mixer Architecture with
Bayesian Network (MABN) to explore the node-wise zero-cost statistics and
estimate node-specific uncertainty. Moreover, we propose DiffKendall as a loss
function to directly optimize Kendall's Tau coefficient in a differentiable
manner so that our ParZC can better handle the discrepancies in ranking
architectures. Comprehensive experiments on NAS-Bench-101, 201, and NDS
demonstrate the superiority of our proposed ParZC compared to existing
zero-shot NAS methods. Additionally, we demonstrate the versatility and
adaptability of ParZC by transferring it to the Vision Transformer search
space.

提出了一种新颖的 Parametric Zero-Cost Proxies (ParZC) 框架，通过参数化来增强零成本代理的适应性，在节点的零成本统计中探索并估计节点特定的不确定性，并提出了 DiffKendall 作为一种损失函数，以不可微分的方式直接优化 Kendall's Tau 系数，从而更好地处理架构排名中的差异。在 NAS-Bench-101、201 和 NDS 上的综合实验证明了我们提出的 ParZC 相对于现有的零成本 NAS 方法的优越性，并通过将其应用于 Vision Transformer 搜索空间展示了 ParZC 的多功能性和适应性。