Meta-learning, or learning to learn, is a machine learning approach that
utilizes prior learning experiences to expedite the learning process on unseen
tasks. As a data-driven approach, meta-learning requires meta-features that
represent the primary learning tasks or datasets, and are estimated
traditonally as engineered dataset statistics that require expert domain
knowledge tailored for every meta-task. In this paper, first, we propose a
meta-feature extractor called Dataset2Vec that combines the versatility of
engineered dataset meta-features with the expressivity of meta-features learned
by deep neural networks. Primary learning tasks or datasets are represented as
hierarchical sets, i.e., as a set of sets, esp. as a set of predictor/target
pairs, and then a DeepSet architecture is employed to regress meta-features on
them. Second, we propose a novel auxiliary meta-learning task with abundant
data called dataset similarity learning that aims to predict if two batches
stem from the same dataset or different ones. In an experiment on a large-scale
hyperparameter optimization task for 120 UCI datasets with varying schemas as a
meta-learning task, we show that the meta-features of Dataset2Vec outperform
the expert engineered meta-features and thus demonstrate the usefulness of
learned meta-features for datasets with varying schemas for the first time.

本文提出了一种基于深层神经网络的元特征提取器 Dataset2Vec 和基于 DeepSet 结构的元学习方法，在大规模数据集优化任务中将元学习应用于不同模式的数据集，实现了性能优于专家工程方法的元特征学习以及元学习辅助任务。