Current state-of-the-art approaches to text classification typically leverage
BERT-style Transformer models with a softmax classifier, jointly fine-tuned to
predict class labels of a target task. In this paper, we instead propose an
alternative training objective in which we learn task-specific embeddings of
text: our proposed objective learns embeddings such that all texts that share
the same target class label should be close together in the embedding space,
while all others should be far apart. This allows us to replace the softmax
classifier with a more interpretable k-nearest-neighbor classification
approach. In a series of experiments, we show that this yields a number of
interesting benefits: (1) The resulting order induced by distances in the
embedding space can be used to directly explain classification decisions. (2)
This facilitates qualitative inspection of the training data, helping us to
better understand the problem space and identify labelling quality issues. (3)
The learned distances to some degree generalize to unseen classes, allowing us
to incrementally add new classes without retraining the model. We present
extensive experiments which show that the benefits of ante-hoc explainability
and incremental learning come at no cost in overall classification accuracy,
thus pointing to practical applicability of our proposed approach.

本文提出了一种基于 k 最近邻分类的替代方法，学习任务特定的文本嵌入表示方式，从而实现解释性和增量学习，而不影响分类准确性。