Exploiting label hierarchies has become a promising approach to tackling the
zero-shot multi-label text classification (ZS-MTC) problem. Conventional
methods aim to learn a matching model between text and labels, using a graph
encoder to incorporate label hierarchies to obtain effective label
representations \cite{rios2018few}. More recently, pretrained models like BERT
\cite{devlin2018bert} have been used to convert classification tasks into a
textual entailment task \cite{yin-etal-2019-benchmarking}. This approach is
naturally suitable for the ZS-MTC task. However, pretrained models are
underexplored in the existing work because they do not generate individual
vector representations for text or labels, making it unintuitive to combine
them with conventional graph encoding methods. In this paper, we explore to
improve pretrained models with label hierarchies on the ZS-MTC task. We propose
a Reinforced Label Hierarchy Reasoning (RLHR) approach to encourage
interdependence among labels in the hierarchies during training. Meanwhile, to
overcome the weakness of flat predictions, we design a rollback algorithm that
can remove logical errors from predictions during inference. Experimental
results on three real-life datasets show that our approach achieves better
performance and outperforms previous non-pretrained methods on the ZS-MTC task.

本研究提出了一种加强预训练模型和标签层次结构的方法（RLHR）来优化 ZS-MTC 任务，并设计了一个回滚算法来消除推理期间的逻辑错误，实验结果显示我们的方法在三个真实数据集上表现更好，优于之前的非预训练方法。

强化标签层次推理提升预训练模型在零样本多标签文本分类中的表现

Improving Pretrained Models for Zero-shot Multi-label Text  Classification through Reinforced Label Hierarchy Reasoning

Different from the traditional classification tasks which assume mutual
exclusion of labels, hierarchical multi-label classification (HMLC) aims to
assign multiple labels to every instance with the labels organized under
hierarchical relations. Besides the labels, since linguistic ontologies are
intrinsic hierarchies, the conceptual relations between words can also form
hierarchical structures. Thus it can be a challenge to learn mappings from word
hierarchies to label hierarchies. We propose to model the word and label
hierarchies by embedding them jointly in the hyperbolic space. The main reason
is that the tree-likeness of the hyperbolic space matches the complexity of
symbolic data with hierarchical structures. A new Hyperbolic Interaction Model
(HyperIM) is designed to learn the label-aware document representations and
make predictions for HMLC. Extensive experiments are conducted on three
benchmark datasets. The results have demonstrated that the new model can
realistically capture the complex data structures and further improve the
performance for HMLC comparing with the state-of-the-art methods. To facilitate
future research, our code is publicly available.

本文提出了一种使用超几何空间嵌入词汇和标签层次结构的方法来解决层级多标签分类问题。通过设计新的超球面交互模型，实现对标签感知的文档表示和 HMLC 预测，实验结果表明新模型相对于现有方法进一步提高了性能。