Feature Selection (FS), such as filter, wrapper, and embedded methods, aims
to find the optimal feature subset for a given downstream task. However, in
many real-world practices, 1) the criteria of FS vary across domains; 2) FS is
brittle when data is a high-dimensional and small sample size. Can selected
feature subsets be more generalized, accurate, and input dimensionality
agnostic? We generalize this problem into a deep differentiable feature
selection task and propose a new perspective: discrete feature subsetting as
continuous embedding space optimization. We develop a generic and principled
framework including a deep feature subset encoder, accuracy evaluator, decoder,
and gradient ascent optimizer. This framework implements four steps: 1)
features-accuracy training data preparation; 2) deep feature subset embedding;
3) gradient-optimized search; 4) feature subset reconstruction. We develop new
technical insights: reinforcement as a training data generator, ensembles of
diverse peer and exploratory feature selector knowledge for generalization, an
effective embedding from feature subsets to continuous space along with joint
optimizing reconstruction and accuracy losses to select accurate features.
Experimental results demonstrate the effectiveness of the proposed method.

通过将离散特征子集转换为连续嵌入空间优化的方法，我们提出了一种新的深度可微特征选择任务，实现了一种通用和原则性的框架，该框架包括深度特征子集编码器，准确性评估器，解码器和梯度上升优化器。

数据中心的 AI：利用离散子集作为连续嵌入空间优化的深度生成可微特征选择

Data-Centric AI: Deep Generative Differentiable Feature Selection via Discrete Subsetting as Continuous Embedding Space Optimization

Feature selection is a crucial step in developing robust and powerful machine
learning models. Feature selection techniques can be divided into two
categories: filter and wrapper methods. While wrapper methods commonly result
in strong predictive performances, they suffer from a large computational
complexity and therefore take a significant amount of time to complete,
especially when dealing with high-dimensional feature sets. Alternatively,
filter methods are considerably faster, but suffer from several other
disadvantages, such as (i) requiring a threshold value, (ii) not taking into
account intercorrelation between features, and (iii) ignoring feature
interactions with the model. To this end, we present powershap, a novel wrapper
feature selection method, which leverages statistical hypothesis testing and
power calculations in combination with Shapley values for quick and intuitive
feature selection. Powershap is built on the core assumption that an
informative feature will have a larger impact on the prediction compared to a
known random feature. Benchmarks and simulations show that powershap
outperforms other filter methods with predictive performances on par with
wrapper methods while being significantly faster, often even reaching half or a
third of the execution time. As such, powershap provides a competitive and
quick algorithm that can be used by various models in different domains.
Furthermore, powershap is implemented as a plug-and-play and open-source
sklearn component, enabling easy integration in conventional data science
pipelines. User experience is even further enhanced by also providing an
automatic mode that automatically tunes the hyper-parameters of the powershap
algorithm, allowing to use the algorithm without any configuration needed.

本论文提出了一种基于统计假设检验和 Shapley 值的新型包装特征选择方法 powershap，通过对预测结果的影响进行快速且直观的特征选择。该方法速度更快且价格更具竞争力，并且由于实现为 sklearn 组件，可以轻松集成到数据科学流程中。