The field of imbalanced self-supervised learning, especially in the context
of tabular data, has not been extensively studied. Existing research has
predominantly focused on image datasets. This paper aims to fill this gap by
examining the specific challenges posed by data imbalance in self-supervised
learning in the domain of tabular data, with a primary focus on autoencoders.
Autoencoders are widely employed for learning and constructing a new
representation of a dataset, particularly for dimensionality reduction. They
are also often used for generative model learning, as seen in variational
autoencoders. When dealing with mixed tabular data, qualitative variables are
often encoded using a one-hot encoder with a standard loss function (MSE or
Cross Entropy). In this paper, we analyze the drawbacks of this approach,
especially when categorical variables are imbalanced. We propose a novel metric
to balance learning: a Multi-Supervised Balanced MSE. This approach reduces the
reconstruction error by balancing the influence of variables. Finally, we
empirically demonstrate that this new metric, compared to the standard MSE: i)
outperforms when the dataset is imbalanced, especially when the learning
process is insufficient, and ii) provides similar results in the opposite case.

通过研究表格数据中自我监督学习不平衡，尤其是在图像数据集中，看到现有研究主要集中在图像数据集，本文旨在填补这一空白，着重探讨自我监督学习在表格数据领域中因数据不平衡而带来的具体挑战，重点关注自编码器。我们分析了目前使用一个热编码器和标准损失函数（MSE 或交叉熵）对混合表格数据进行编码的方法的缺点，特别是在分类变量不平衡时。我们提出了一种新的平衡学习的度量方法：多重监督平衡 MSE。这种方法通过平衡变量的影响来减少重构误差。最后，我们通过实验证明了这种新度量方法相比于标准 MSE，在数据集不平衡且学习过程不充分时表现更好，并在相反情况下提供类似的结果。