Integrating ML models in software is of growing interest. Building accurate models requires right choice of hyperparameters for training procedures (learners), when the training dataset is given. AutoML tools provide APIs to automate the choice, which usually involve many trials of different hyperparameters for a given training dataset. Since training and evaluation of complex models can be time and resource consuming, existing AutoML solutions require long time or large resource to produce accurate models for large scale training data. That prevents AutoML to be embedded in a software which needs to repeatedly tune hyperparameters and produce models to be consumed by other components, such as large-scale data systems. We present a fast and lightweight hyperparameter optimization method FLO and use it to build an efficient AutoML solution. Our method optimizes for minimal evaluation cost instead of number of iterations to find accurate models. Our main idea is to leverage a holistic consideration of the relations among model complexity, evaluation cost and accuracy. FLO has a strong anytime performance and significantly outperforms Bayesian Optimization and random search for hyperparameter tuning on a large open source AutoML Benchmark. Our AutoML solution also outperforms top-ranked AutoML libraries in a majority of the tasks on this benchmark.

研究使用低计算成本自动选择学习器和超参数的问题，FLAML是一个快速而轻量级的库，它能够根据给定的训练数据集和误差指标来优化计算资源的使用，同时在多个方面对试验成本和模型误差进行了研究并提出了设计指南，并整合了多种简单而有效的搜索策略，在相同或更少的预算限制下在大型开源AutoML基准测试中显著优于排名前列的AutoML库。

FLAML: 一个快速轻量级的AutoML库