Since most machine learning (ML) algorithms are designed for numerical
inputs, efficiently encoding categorical variables is a crucial aspect in data
analysis. A common problem are high cardinality features, i.e. unordered
categorical predictor variables with a high number of levels. We study
techniques that yield numeric representations of categorical variables which
can then be used in subsequent ML applications. We focus on the impact of these
techniques on a subsequent algorithm's predictive performance, and -- if
possible -- derive best practices on when to use which technique. We conducted
a large-scale benchmark experiment, where we compared different encoding
strategies together with five ML algorithms (lasso, random forest, gradient
boosting, k-nearest neighbors, support vector machine) using datasets from
regression, binary- and multiclass- classification settings. In our study,
regularized versions of target encoding (i.e. using target predictions based on
the feature levels in the training set as a new numerical feature) consistently
provided the best results. Traditionally widely used encodings that make
unreasonable assumptions to map levels to integers (e.g. integer encoding) or
to reduce the number of levels (possibly based on target information, e.g. leaf
encoding) before creating binary indicator variables (one-hot or dummy
encoding) were not as effective in comparison.

本文基于机器学习算法对分类变量进行编码技术探究，通过对比不同编码策略和算法的实验结果，发现对训练数据中的特征进行正则化后的目标编码技术能够提供最好的结果，并且传统编码技术在部分情况下不如目标编码技术。