Deep Neural Networks (DNN) have demonstrated superior ability to extract high
level embedding vectors from low level features. Despite the success, the
serving time is still the bottleneck due to expensive run-time computation of
multiple layers of dense matrices. GPGPU, FPGA, or ASIC-based serving systems
require additional hardware that are not in the mainstream design of most
commercial applications. In contrast, tree or forest-based models are widely
adopted because of low serving cost, but heavily depend on carefully engineered
features. This work proposes a Deep Embedding Forest model that benefits from
the best of both worlds. The model consists of a number of embedding layers and
a forest/tree layer. The former maps high dimensional (hundreds of thousands to
millions) and heterogeneous low-level features to the lower dimensional
(thousands) vectors, and the latter ensures fast serving.
Built on top of a representative DNN model called Deep Crossing, and two
forest/tree-based models including XGBoost and LightGBM, a two-step Deep
Embedding Forest algorithm is demonstrated to achieve on-par or slightly better
performance as compared with the DNN counterpart, with only a fraction of
serving time on conventional hardware. After comparing with a joint
optimization algorithm called partial fuzzification, also proposed in this
paper, it is concluded that the two-step Deep Embedding Forest has achieved
near optimal performance. Experiments based on large scale data sets (up to 1
billion samples) from a major sponsored search engine proves the efficacy of
the proposed model.

该论文提出了一种名为 Deep Embedding Forest 的模型，采用嵌入层和基于树 / 森林的层，以实现快速服务支持，同时借鉴 Deep Neural Networks 的高维特征映射能力，通过仅使用传统硬件，该模型在性能上与 DNN 模型相当。