A feature selection algorithm should ideally satisfy four conditions:
reliably extract relevant features; be able to identify non-linear feature
interactions; scale linearly with the number of features and dimensions; allow
the incorporation of known sparsity structure. In this work we propose a novel
feature selection algorithm, Gradient Boosted Feature Selection (GBFS), which
satisfies all four of these requirements. The algorithm is flexible, scalable,
and surprisingly straight-forward to implement as it is based on a modification
of Gradient Boosted Trees. We evaluate GBFS on several real world data sets and
show that it matches or out-performs other state of the art feature selection
algorithms. Yet it scales to larger data set sizes and naturally allows for
domain-specific side information.

本研究提出了一种名为梯度提升特征选择 (GBFS) 的新型特征选择算法，该算法可可靠地提取相关特征，能够识别非线性特征相互作用，与特征和维度的数量线性缩放，并允许纳入已知的稀疏结构，并在多个真实世界的数据集上进行了评估，表现出强大的性能，比其他现有的特征选择算法具有更好的扩展性和适应性。

梯度提升特征选择

Gradient Boosted Feature Selection

Many predictive tasks of web applications need to model categorical
variables, such as user IDs and demographics like genders and occupations. To
apply standard machine learning techniques, these categorical predictors are
always converted to a set of binary features via one-hot encoding, making the
resultant feature vector highly sparse. To learn from such sparse data
effectively, it is crucial to account for the interactions between features.
Factorization Machines (FMs) are a popular solution for efficiently using the
second-order feature interactions. However, FM models feature interactions in a
linear way, which can be insufficient for capturing the non-linear and complex
inherent structure of real-world data. While deep neural networks have recently
been applied to learn non-linear feature interactions in industry, such as the
Wide&Deep by Google and DeepCross by Microsoft, the deep structure meanwhile
makes them difficult to train.
In this paper, we propose a novel model Neural Factorization Machine (NFM)
for prediction under sparse settings. NFM seamlessly combines the linearity of
FM in modelling second-order feature interactions and the non-linearity of
neural network in modelling higher-order feature interactions. Conceptually,
NFM is more expressive than FM since FM can be seen as a special case of NFM
without hidden layers. Empirical results on two regression tasks show that with
one hidden layer only, NFM significantly outperforms FM with a 7.3% relative
improvement. Compared to the recent deep learning methods Wide&Deep and
DeepCross, our NFM uses a shallower structure but offers better performance,
being much easier to train and tune in practice.

本文提出了一种用于稀疏数据预测的新型模型 Neural Factorization Machine（NFM），将因子分解机（Factorization Machines，FM）中二阶特征交互的线性性与神经网络中高阶特征交互的非线性相结合，实验证明 NFM 在深度学习方法 Wide&Deep 和 DeepCross 的基础上，使用更浅的结构却提供更好的性能，在实际中更易于训练和调节。