Feature selection is an important process in machine learning. It builds an
interpretable and robust model by selecting the features that contribute the
most to the prediction target. However, most mature feature selection
algorithms, including supervised and semi-supervised, fail to fully exploit the
complex potential structure between features. We believe that these structures
are very important for the feature selection process, especially when labels
are lacking and data is noisy.
To this end, we innovatively introduce a deep learning-based self-supervised
mechanism into feature selection problems, namely batch-Attention-based
Self-supervision Feature Selection(A-SFS). Firstly, a multi-task
self-supervised autoencoder is designed to uncover the hidden structure among
features with the support of two pretext tasks. Guided by the integrated
information from the multi-self-supervised learning model, a batch-attention
mechanism is designed to generate feature weights according to batch-based
feature selection patterns to alleviate the impacts introduced by a handful of
noisy data. This method is compared to 14 major strong benchmarks, including
LightGBM and XGBoost. Experimental results show that A-SFS achieves the highest
accuracy in most datasets. Furthermore, this design significantly reduces the
reliance on labels, with only 1/10 labeled data needed to achieve the same
performance as those state of art baselines. Results show that A-SFS is also
most robust to the noisy and missing data.

通过引入基于深度学习的自监督机制，提出了一种批量注意力自监督特征选择（A-SFS）方法，该方法利用多任务自监督自编码器发现特征之间的隐藏结构，并通过批量注意力机制生成特征权重，以减少少量有噪音数据的影响。与 14 种主要的算法进行比较，实验结果表明，该方法在大多数数据集中具有最高的准确性，并且在降低对标签的依赖，并且对于噪声和丢失数据最具有鲁棒性。