Recent advances in Neural Architecture Search (NAS) have produced
state-of-the-art architectures on several tasks. NAS shifts the efforts of
human experts from developing novel architectures directly to designing
architecture search spaces and methods to explore them efficiently. The search
space definition captures prior knowledge about the properties of the
architectures and it is crucial for the complexity and the performance of the
search algorithm. However, different search space definitions require
restarting the learning process from scratch. We propose a novel agent based on
the Transformer that supports joint training and efficient transfer of prior
knowledge between multiple search spaces and tasks.

该论文提出了一种基于 Transformer 的新型代理，支持多个搜索空间和任务之间的联合训练和高效先前知识的传输。

Transfer NAS: 基于 Transformer 代理的搜索空间知识迁移

Transfer NAS: Knowledge Transfer between Search Spaces with Transformer  Agents

Thanks to the recent development of deep generative models, it is becoming
possible to generate high-quality images with both fidelity and diversity.
However, the training of such generative models requires a large dataset. To
reduce the amount of data required, we propose a new method for transferring
prior knowledge of the pre-trained generator, which is trained with a large
dataset, to a small dataset in a different domain. Using such prior knowledge,
the model can generate images leveraging some common sense that cannot be
acquired from a small dataset. In this work, we propose a novel method focusing
on the parameters for batch statistics, scale and shift, of the hidden layers
in the generator. By training only these parameters in a supervised manner, we
achieved stable training of the generator, and our method can generate higher
quality images compared to previous methods without collapsing, even when the
dataset is small (~100). Our results show that the diversity of the filters
acquired in the pre-trained generator is important for the performance on the
target domain. Our method makes it possible to add a new class or domain to a
pre-trained generator without disturbing the performance on the original
domain.

利用预训练生成器的先前知识，通过对生成器中的批次统计参数进行有监督训练，实现了在小数据集下生成高质量图像的新方法。