This paper presents a framework in which hierarchical softmax is used to
create a global hierarchical classifier. The approach is applicable for any
classification task where there is a natural hierarchy among classes. We show
empirical results on four text classification datasets. In all datasets the
hierarchical softmax improved on the regular softmax used in a flat classifier
in terms of macro-F1 and macro-recall. In three out of four datasets
hierarchical softmax achieved a higher micro-accuracy and macro-precision.

该论文提出了一个框架，在其中使用分层 softmax 来创建一个全局的分层分类器。该方法适用于任何具有类别的自然层次结构的分类任务。我们在四个文本分类数据集上展示了实证结果。在所有的数据集中，相比于在扁平分类器中使用的常规 softmax，分层 softmax 在宏 F1 和宏召回上有显著的改进。在四个数据集中的三个中，分层 softmax 实现了更高的微精确度和宏精确度。

全球级分层神经网络使用层级 Softmax

Global Hierarchical Neural Networks using Hierarchical Softmax

Typically, Softmax is used in the final layer of a neural network to get a
probability distribution for output classes. But the main problem with Softmax
is that it is computationally expensive for large scale data sets with large
number of possible outputs. To approximate class probability efficiently on
such large scale data sets we can use Hierarchical Softmax. LSHTC datasets were
used to study the performance of the Hierarchical Softmax. LSHTC datasets have
large number of categories. In this paper we evaluate and report the
performance of normal Softmax Vs Hierarchical Softmax on LSHTC datasets. This
evaluation used macro f1 score as a performance measure. The observation was
that the performance of Hierarchical Softmax degrades as the number of classes
increase.

本文比较了 Softmax 和 Hierarchical Softmax 对于 LSHTC 数据集的性能表现，结果发现在类别数量较多时，Hierarchical Softmax 的表现会下降。

层级 softmax 在大规模分类任务中的有效性

Effectiveness of Hierarchical Softmax in Large Scale Classification  Tasks

Extreme multi-label classification (XMLC) is a problem of tagging an instance
with a small subset of relevant labels chosen from an extremely large pool of
possible labels. Large label spaces can be efficiently handled by organizing
labels as a tree, like in the hierarchical softmax (HSM) approach commonly used
for multi-class problems. In this paper, we investigate probabilistic label
trees (PLTs) that have been recently devised for tackling XMLC problems. We
show that PLTs are a no-regret multi-label generalization of HSM when
precision@k is used as a model evaluation metric. Critically, we prove that
pick-one-label heuristic - a reduction technique from multi-label to
multi-class that is routinely used along with HSM - is not consistent in
general. We also show that our implementation of PLTs, referred to as
extremeText (XT), obtains significantly better results than HSM with the
pick-one-label heuristic and XML-CNN, a deep network specifically designed for
XMLC problems. Moreover, XT is competitive to many state-of-the-art approaches
in terms of statistical performance, model size and prediction time which makes
it amenable to deploy in an online system.

本文中，我们探究了处理极端多标记分类问题的概率标记树（PLTs），并通过实现 extremeText（XT）来证明 PLTs 在 precision@k 指标下是一个 HSM 的无遗憾多标签泛化。同时，我们证明了 pick-one-label 启发式算法在一般情况下不是一致的，我们的实现方法比使用 pick-one-label 的 HSM 和专门为 XMLC 问题设计的神经网络 XML-CNN 得到的结果更好。此外，XT 在统计性能、模型大小和预测时间等方面与许多最先进的方法相当，适合部署在在线系统中。

无后悔泛化的分层 softmax 算法在极端多标签分类中的应用

A no-regret generalization of hierarchical softmax to extreme  multi-label classification

Training neural network language models over large vocabularies is still
computationally very costly compared to count-based models such as Kneser-Ney.
At the same time, neural language models are gaining popularity for many
applications such as speech recognition and machine translation whose success
depends on scalability. We present a systematic comparison of strategies to
represent and train large vocabularies, including softmax, hierarchical
softmax, target sampling, noise contrastive estimation and self normalization.
We further extend self normalization to be a proper estimator of likelihood and
introduce an efficient variant of softmax. We evaluate each method on three
popular benchmarks, examining performance on rare words, the speed/accuracy
trade-off and complementarity to Kneser-Ney.

本研究系统比较了代表性的方法来表示和训练大词汇下的神经网络语言模型，包括 softmax、层级 softmax、自归一化等方法，并在三个流行的基准数据集上评估了每种方法在罕见词汇、速度 / 精度平衡以及与 Kneser-Ney 模型之间的互补性方面的表现。