In this work, we propose a novel tree-based explanation technique, PEACH
(Pretrained-embedding Explanation Across Contextual and Hierarchical
Structure), that can explain how text-based documents are classified by using
any pretrained contextual embeddings in a tree-based human-interpretable
manner. Note that PEACH can adopt any contextual embeddings of the PLMs as a
training input for the decision tree. Using the proposed PEACH, we perform a
comprehensive analysis of several contextual embeddings on nine different NLP
text classification benchmarks. This analysis demonstrates the flexibility of
the model by applying several PLM contextual embeddings, its attribute
selections, scaling, and clustering methods. Furthermore, we show the utility
of explanations by visualising the feature selection and important trend of
text classification via human-interpretable word-cloud-based trees, which
clearly identify model mistakes and assist in dataset debugging. Besides
interpretability, PEACH outperforms or is similar to those from pretrained
models.

我们提出了一种新颖的基于树结构的解释技术 PEACH，它可以使用任何预训练的上下文嵌入以人可解释的方式解释文本文档的分类。使用 PEACH，我们对九个不同的 NLP 文本分类基准进行了全面分析，并通过人可解释的词云树来可视化特征选择和文本分类的重要趋势，同时明确了模型的错误并辅助数据集调试。除了解释性外，PEACH 在性能上优于或与预训练模型相似。

PEACH: 预训练嵌入在上下文和层次结构中解释

PEACH: Pretrained-embedding Explanation Across Contextual and  Hierarchical Structure

Despite the extensive amount of labeled datasets in the NLP text
classification field, the persistent imbalance in data availability across
various languages remains evident. Ukrainian, in particular, stands as a
language that still can benefit from the continued refinement of cross-lingual
methodologies. Due to our knowledge, there is a tremendous lack of Ukrainian
corpora for typical text classification tasks. In this work, we leverage the
state-of-the-art advances in NLP, exploring cross-lingual knowledge transfer
methods avoiding manual data curation: large multilingual encoders and
translation systems, LLMs, and language adapters. We test the approaches on
three text classification tasks -- toxicity classification, formality
classification, and natural language inference -- providing the "recipe" for
the optimal setups.

我们利用 NLP 的最新进展，探索跨语言知识转移方法，避免手动数据整理，以改善 NLP 文本分类领域中各种语言之间数据可用性的不平衡问题，并在三个文本分类任务上进行测试，提供最佳设置的 “配方”。