This work presents a novel approach to neural architecture search (NAS) that
aims to increase carbon efficiency for the model design process. The proposed
framework CE-NAS addresses the key challenge of high carbon cost associated
with NAS by exploring the carbon emission variations of energy and energy
differences of different NAS algorithms. At the high level, CE-NAS leverages a
reinforcement-learning agent to dynamically adjust GPU resources based on
carbon intensity, predicted by a time-series transformer, to balance
energy-efficient sampling and energy-intensive evaluation tasks. Furthermore,
CE-NAS leverages a recently proposed multi-objective optimizer to effectively
reduce the NAS search space. We demonstrate the efficacy of CE-NAS in lowering
carbon emissions while achieving SOTA results for both NAS datasets and
open-domain NAS tasks. For example, on the HW-NasBench dataset, CE-NAS reduces
carbon emissions by up to 7.22X while maintaining a search efficiency
comparable to vanilla NAS. For open-domain NAS tasks, CE-NAS achieves SOTA
results with 97.35% top-1 accuracy on CIFAR-10 with only 1.68M parameters and a
carbon consumption of 38.53 lbs of CO2. On ImageNet, our searched model
achieves 80.6% top-1 accuracy with a 0.78 ms TensorRT latency using FP16 on
NVIDIA V100, consuming only 909.86 lbs of CO2, making it comparable to other
one-shot-based NAS baselines.

该论文提出了一种新颖的神经架构搜索（NAS）方法，旨在增加模型设计过程的碳效率。CE-NAS 框架通过探索不同 NAS 算法的碳排放变化和能量差异来解决与 NAS 相关的高碳成本的关键挑战。在高层次上，CE-NAS 利用增强学习代理根据时间序列变压器预测的碳强度动态调整 GPU 资源，以平衡高能效采样和高能耗评估任务。此外，CE-NAS 还利用最近提出的多目标优化器有效减少了 NAS 的搜索空间。我们在降低碳排放的同时，在 NAS 数据集和开放域 NAS 任务上实现了 SOTA 结果的有效性。例如，在 HW-NasBench 数据集上，CE-NAS 将碳排放减少了最多 7.22 倍，同时保持了与原始 NAS 相当的搜索效率。对于开放域 NAS 任务，CE-NAS 在 CIFAR-10 上以仅有 1.68M 参数和 38.53 磅二氧化碳的碳耗的情况下达到了 97.35％ 的 top-1 精度。在 ImageNet 上，我们的搜索模型在 NVIDIA V100 上使用 FP16 和 0.78 毫秒的 TensorRT 延迟，仅消耗 909.86 磅二氧化碳，与其他基于一次性采样的 NAS 基线相当。