Feature selection aims to identify the optimal feature subset for enhancing
downstream models. Effective feature selection can remove redundant features,
save computational resources, accelerate the model learning process, and
improve the model overall performance. However, existing works are often
time-intensive to identify the effective feature subset within high-dimensional
feature spaces. Meanwhile, these methods mainly utilize a single downstream
task performance as the selection criterion, leading to the selected subsets
that are not only redundant but also lack generalizability. To bridge these
gaps, we reformulate feature selection through a neuro-symbolic lens and
introduce a novel generative framework aimed at identifying short and effective
feature subsets. More specifically, we found that feature ID tokens of the
selected subset can be formulated as symbols to reflect the intricate
correlations among features. Thus, in this framework, we first create a data
collector to automatically collect numerous feature selection samples
consisting of feature ID tokens, model performance, and the measurement of
feature subset redundancy. Building on the collected data, an
encoder-decoder-evaluator learning paradigm is developed to preserve the
intelligence of feature selection into a continuous embedding space for
efficient search. Within the learned embedding space, we leverage a
multi-gradient search algorithm to find more robust and generalized embeddings
with the objective of improving model performance and reducing feature subset
redundancy. These embeddings are then utilized to reconstruct the feature ID
tokens for executing the final feature selection. Ultimately, comprehensive
experiments and case studies are conducted to validate the effectiveness of the
proposed framework.

本研究通过神经符号化的视角提出了一个新颖的生成框架，旨在通过学习有效特征子集来改进下游模型的性能，并且提出了一个基于编码器 - 解码器 - 评估器学习范式，在连续嵌入空间中保留特征选择的智能进行高效搜索。综合实验证实了该框架的有效性。