Categorical variables often appear in datasets for classification and
regression tasks, and they need to be encoded into numerical values before
training. Since many encoders have been developed and can significantly impact
performance, choosing the appropriate encoder for a task becomes a
time-consuming yet important practical issue. This study broadly classifies
machine learning models into three categories: 1) ATI models that implicitly
perform affine transformations on inputs, such as multi-layer perceptron neural
network; 2) Tree-based models that are based on decision trees, such as random
forest; and 3) the rest, such as kNN. Theoretically, we prove that the one-hot
encoder is the best choice for ATI models in the sense that it can mimic any
other encoders by learning suitable weights from the data. We also explain why
the target encoder and its variants are the most suitable encoders for
tree-based models. This study conducted comprehensive computational experiments
to evaluate 14 encoders, including one-hot and target encoders, along with
eight common machine-learning models on 28 datasets. The computational results
agree with our theoretical analysis. The findings in this study shed light on
how to select the suitable encoder for data scientists in fields such as fraud
detection, disease diagnosis, etc.

本研究对 14 种编码器以及八种常见机器学习模型在 28 个数据集上进行了全面的实验评估，发现了不同编码器在不同情境下的最佳选择，并为数据科学家在欺诈检测、疾病诊断等领域提供了选择合适编码器的指导。