基于强化学习的神经优化器搜索

Sep, 2017

Neural Optimizer Search with Reinforcement Learning

Irwan Bello, Barret Zoph, Vijay Vasudevan, Quoc V. Le

TL;DR用循环神经网络控制器生成包括多种基本函数的特定领域语言的更新方程，通过强化学习来优化模型的性能，在CIFAR-10上发现了多种优化器，新增的PowerSign和AddSign优化器能够提高不同任务和架构的训练。

Abstract

We present an approach to automate the process of discovering optimization methods, with a focus on deep learning architectures. We train a Recurrent Neural Network controller to generate a string in a domain spe