The tuning of hyperparameters becomes increasingly important as machine learning (ML) models have been extensively applied in data mining applications. Among various approaches, Bayesian optimization (BO) is a successful methodology to tune hyper-parameters automatically. While traditional methods optimize each tuning task in isolation, there has been recent interest in speeding up BO by transferring knowledge across previous tasks. In this work, we introduce an automatic method to design the BO search space with the aid of tuning history from past tasks. This simple yet effective approach can be used to endow many existing BO methods with transfer learning capabilities. In addition, it enjoys the three advantages: universality, generality, and safeness. The extensive experiments show that our approach considerably boosts BO by designing a promising and compact search space instead of using the entire space, and outperforms the state-of-the-arts on a wide range of benchmarks, including machine learning and deep learning tuning tasks, and neural architecture search.

本论文提出了一种自动化的基于先前任务调优历史知识的方法，通过传递学习能力将现有的贝叶斯优化方法能够具有迁移学习的能力。同时，作者通过精简搜索空间设计的方法有效地提高了基于机器学习和深度学习，包括神经架构搜索等任务的效能，并在各种基准测试中优于现有技术。

基于迁移学习的超参数调优搜索空间设计