Nowadays, many modern applications require heterogeneous tabular data, which
is still a challenging task in terms of regression and classification. Many
approaches have been proposed to adapt neural networks for this task, but
still, boosting and bagging of decision trees are the best-performing methods
for this task. In this paper, we show that a binomial initialized neural
network can be used effectively on tabular data. The proposed approach shows a
simple but effective approach for initializing the first hidden layer in neural
networks. We also show that this initializing schema can be used to jointly
train ensembles by adding gradient masking to batch entries and using the
binomial initialization for the last layer in a neural network. For this
purpose, we modified the hinge binary loss and the soft max loss to make them
applicable for joint ensemble training. We evaluate our approach on multiple
public datasets and showcase the improved performance compared to other neural
network-based approaches. In addition, we discuss the limitations and possible
further research of our approach for improving the applicability of neural
networks to tabular data.
Link:
this https URL&mode=list

现代应用程序对异构表格数据的回归和分类仍然是一个具有挑战性的任务，本文提出了一种使用二项式初始化的神经网络来有效处理表格数据的方法，同时实现了联合集成训练，并在多个公共数据集上的实验证明了其相较于其他基于神经网络的方法具有改进的性能。

基于表格数据的神经网络的初始化模式

An Initialization Schema for Neuronal Networks on Tabular Data

Graph neural networks (GNNs) are powerful models that have been successful in
various graph representation learning tasks. Whereas gradient boosted decision
trees (GBDT) often outperform other machine learning methods when faced with
heterogeneous tabular data. But what approach should be used for graphs with
tabular node features? Previous GNN models have mostly focused on networks with
homogeneous sparse features and, as we show, are suboptimal in the
heterogeneous setting. In this work, we propose a novel architecture that
trains GBDT and GNN jointly to get the best of both worlds: the GBDT model
deals with heterogeneous features, while GNN accounts for the graph structure.
Our model benefits from end-to-end optimization by allowing new trees to fit
the gradient updates of GNN. With an extensive experimental comparison to the
leading GBDT and GNN models, we demonstrate a significant increase in
performance on a variety of graphs with tabular features. The code is
available: this https URL

该论文提出了一种新的网络结构，将 GNN 和 GBDT 训练在一起，以达到异质图表节点特征下最优化的效果，该模型经过广泛实验，证明在各种具有表格特征的图上性能显著提高。