Because of the inevitable cost and complexity of transformer and pre-trained
models, efficiency concerns are raised for long text classification. Meanwhile,
in the highly sensitive domains, e.g., healthcare and legal long-text mining,
potential model distrust, yet underrated and underexplored, may hatch vital
apprehension. Existing methods generally segment the long text, encode each
piece with the pre-trained model, and use attention or RNNs to obtain long text
representation for classification. In this work, we propose a simple but
effective model, Segment-aWare multIdimensional PErceptron (SWIPE), to replace
attention/RNNs in the above framework. Unlike prior efforts, SWIPE can
effectively learn the label of the entire text with supervised training, while
perceive the labels of the segments and estimate their contributions to the
long-text labeling in an unsupervised manner. As a general classifier, SWIPE
can endorse different encoders, and it outperforms SOTA models in terms of
classification accuracy and model efficiency. It is noteworthy that SWIPE
achieves superior interpretability to transparentize long text classification
results.

该研究提出了一种名为 SWIPE 的模型，它可以有效地学习整个文本的标签，并以一种无监督的方式感知段落的标签并估计它们对文本标签的贡献，从而在长文本分类中实现了高准确度和模型解释性的突破。