Large-scale Multi-label Text Classification (LMTC) has a wide range of
Natural Language Processing (NLP) applications and presents interesting
challenges. First, not all labels are well represented in the training set, due
to the very large label set and the skewed label distributions of LMTC
datasets. Also, label hierarchies and differences in human labelling guidelines
may affect graph-aware annotation proximity. Finally, the label hierarchies are
periodically updated, requiring LMTC models capable of zero-shot
generalization. Current state-of-the-art LMTC models employ Label-Wise
Attention Networks (LWANs), which (1) typically treat LMTC as flat multi-label
classification; (2) may use the label hierarchy to improve zero-shot learning,
although this practice is vastly understudied; and (3) have not been combined
with pre-trained Transformers (e.g. BERT), which have led to state-of-the-art
results in several NLP benchmarks. Here, for the first time, we empirically
evaluate a battery of LMTC methods from vanilla LWANs to hierarchical
classification approaches and transfer learning, on frequent, few, and
zero-shot learning on three datasets from different domains. We show that
hierarchical methods based on Probabilistic Label Trees (PLTs) outperform
LWANs. Furthermore, we show that Transformer-based approaches outperform the
state-of-the-art in two of the datasets, and we propose a new state-of-the-art
method which combines BERT with LWANs. Finally, we propose new models that
leverage the label hierarchy to improve few and zero-shot learning, considering
on each dataset a graph-aware annotation proximity measure that we introduce.

本研究在多个数据集上评估了多种 Large-scale Multi-label Text Classification 方法，发现基于概率标签树的层次分类方法优于 Label-Wise Attention Networks。另外，结合 Transformer 的方法在两个数据集上实现了优秀的性能，同时提出了一种新的最先进的方法，将 BERT 与 LWANs 相结合。此外，还引入了图形感知注释接近度度量，提出了利用标签层次结构来改进少量和零 - shot 学习的新模型。