The substantial success of Vision Transformer (ViT) in computer vision tasks
is largely attributed to the architecture design. This underscores the
necessity of efficient architecture search for designing better ViTs
automatically. As training-based architecture search methods are
computationally intensive, there is a growing interest in training-free methods
that use zero-cost proxies to score ViTs. However, existing training-free
approaches require expert knowledge to manually design specific zero-cost
proxies. Moreover, these zero-cost proxies exhibit limitations to generalize
across diverse domains. In this paper, we introduce Auto-Prox, an automatic
proxy discovery framework, to address the problem. First, we build the
ViT-Bench-101, which involves different ViT candidates and their actual
performance on multiple datasets. Utilizing ViT-Bench-101, we can evaluate
zero-cost proxies based on their score-accuracy correlation. Then, we represent
zero-cost proxies with computation graphs and organize the zero-cost proxy
search space with ViT statistics and primitive operations. To discover generic
zero-cost proxies, we propose a joint correlation metric to evolve and mutate
different zero-cost proxy candidates. We introduce an elitism-preserve strategy
for search efficiency to achieve a better trade-off between exploitation and
exploration. Based on the discovered zero-cost proxy, we conduct a ViT
architecture search in a training-free manner. Extensive experiments
demonstrate that our method generalizes well to different datasets and achieves
state-of-the-art results both in ranking correlation and final accuracy. Codes
can be found at this https URL

通过自动代理发现框架 Auto-Prox，我们介绍了一种训练免费的方法来进行 Vision Transformer（ViT）的体系结构搜索。广泛的实验表明，我们的方法在不同数据集上都具有很好的泛化能力，并在排名相关性和最终准确性方面取得了最先进的结果。