Effectively representing heterogeneous tabular datasets for meta-learning
remains an open problem. Previous approaches rely on predefined meta-features,
for example, statistical measures or landmarkers. Encoder-based models, such as
Dataset2Vec, allow us to extract significant meta-features automatically
without human intervention. This research introduces a novel encoder-based
representation of tabular datasets implemented within the liltab package
available on GitHub this https URL Our package is based on
an established model for heterogeneous tabular data proposed in [Iwata and
Kumagai, 2020]. The proposed approach employs a different model for encoding
feature relationships, generating alternative representations compared to
existing methods like Dataset2Vec. Both of them leverage the fundamental
assumption of dataset similarity learning. In this work, we evaluate
Dataset2Vec and liltab on two common meta-tasks - representing entire datasets
and hyperparameter optimization warm-start. However, validation on an
independent metaMIMIC dataset highlights the nuanced challenges in
representation learning. We show that general representations may not suffice
for some meta-tasks where requirements are not explicitly considered during
extraction.
[Iwata and Kumagai, 2020] Tomoharu Iwata and Atsutoshi Kumagai. Meta-learning
from Tasks with Heterogeneous Attribute Spaces. In Advances in Neural
Information Processing Systems, 2020.

这项研究提出了一种基于编码器的异构表格数据表示方法，它通过自动提取重要的元特征来代替以往依赖预定义的元特征的方法，该方法在整合数据集和超参数优化暖启动等两个常见元任务中与 Dataset2Vec 和 liltab 在元 MIMIC 数据集上的评估显示了表示学习中细微挑战的存在。

超参数优化中基于编码器的预热方法的再思考

Rethinking of Encoder-based Warm-start Methods in Hyperparameter  Optimization

Meta-learning, or learning to learn, is a machine learning approach that
utilizes prior learning experiences to expedite the learning process on unseen
tasks. As a data-driven approach, meta-learning requires meta-features that
represent the primary learning tasks or datasets, and are estimated
traditonally as engineered dataset statistics that require expert domain
knowledge tailored for every meta-task. In this paper, first, we propose a
meta-feature extractor called Dataset2Vec that combines the versatility of
engineered dataset meta-features with the expressivity of meta-features learned
by deep neural networks. Primary learning tasks or datasets are represented as
hierarchical sets, i.e., as a set of sets, esp. as a set of predictor/target
pairs, and then a DeepSet architecture is employed to regress meta-features on
them. Second, we propose a novel auxiliary meta-learning task with abundant
data called dataset similarity learning that aims to predict if two batches
stem from the same dataset or different ones. In an experiment on a large-scale
hyperparameter optimization task for 120 UCI datasets with varying schemas as a
meta-learning task, we show that the meta-features of Dataset2Vec outperform
the expert engineered meta-features and thus demonstrate the usefulness of
learned meta-features for datasets with varying schemas for the first time.

本文提出了一种基于深层神经网络的元特征提取器 Dataset2Vec 和基于 DeepSet 结构的元学习方法，在大规模数据集优化任务中将元学习应用于不同模式的数据集，实现了性能优于专家工程方法的元特征学习以及元学习辅助任务。