Recent one-shot Neural Architecture Search algorithms rely on training a
hardware-agnostic super-network tailored to a specific task and then extracting
efficient sub-networks for different hardware platforms. Popular approaches
separate the training of super-networks from the search for sub-networks, often
employing predictors to alleviate the computational overhead associated with
search. Additionally, certain methods also incorporate the quantization policy
within the search space. However, while the quantization policy search for
convolutional neural networks is well studied, the extension of these methods
to transformers and especially foundation models remains under-explored. In
this paper, we demonstrate that by using multi-objective search algorithms
paired with lightly trained predictors, we can efficiently search for both the
sub-network architecture and the corresponding quantization policy and
outperform their respective baselines across different performance objectives
such as accuracy, model size, and latency. Specifically, we demonstrate that
our approach performs well across both uni-modal (ViT and BERT) and multi-modal
(BEiT-3) transformer-based architectures as well as convolutional architectures
(ResNet). For certain networks, we demonstrate an improvement of up to $4.80x$
and $3.44x$ for latency and model size respectively, without degradation in
accuracy compared to the fully quantized INT8 baselines.

通过使用多目标搜索算法与轻度训练的预测器相结合，我们展示了在准确性、模型大小和延迟等不同性能目标中，我们的方法在 ViT、BERT、BEiT-3 和 ResNet 等转换器和卷积网络的架构上比各自的基准表现更好，特定网络的性能提升在延迟方面最高达到 4.80 倍，在模型大小方面最高达到 3.44 倍，与全量化 INT8 基准相比准确性不降低。

SimQ-NAS: 同時量化策略與神經架構搜尋

SimQ-NAS: Simultaneous Quantization Policy and Neural Architecture  Search

As the applications of deep learning models on edge devices increase at an
accelerating pace, fast adaptation to various scenarios with varying resource
constraints has become a crucial aspect of model deployment. As a result, model
optimization strategies with adaptive configuration are becoming increasingly
popular. While single-shot quantized neural architecture search enjoys
flexibility in both model architecture and quantization policy, the combined
search space comes with many challenges, including instability when training
the weight-sharing supernet and difficulty in navigating the exponentially
growing search space. Existing methods tend to either limit the architecture
search space to a small set of options or limit the quantization policy search
space to fixed precision policies. To this end, we propose BatchQuant, a robust
quantizer formulation that allows fast and stable training of a compact,
single-shot, mixed-precision, weight-sharing supernet. We employ BatchQuant to
train a compact supernet (offering over $10^{76}$ quantized subnets) within
substantially fewer GPU hours than previous methods. Our approach,
Quantized-for-all (QFA), is the first to seamlessly extend one-shot
weight-sharing NAS supernet to support subnets with arbitrary ultra-low
bitwidth mixed-precision quantization policies without retraining. QFA opens up
new possibilities in joint hardware-aware neural architecture search and
quantization. We demonstrate the effectiveness of our method on ImageNet and
achieve SOTA Top-1 accuracy under a low complexity constraint ($<20$ MFLOPs).
The code and models will be made publicly available at
this https URL

我们提出了 BatchQuant，这是一种稳健的量化器公式，可在数量少得多的 GPU 小时内训练出一种超过 10^{76} 个量化子网的紧凑超网，并首次无需重新训练即可无缝扩展一次权重共享 NAS 超网以支持任意超低位宽混合精度量化策略的子网。