The development of largely human-annotated benchmarks has driven the success
of deep neural networks in various NLP tasks. To enhance the effectiveness of
existing benchmarks, collecting new additional input-output pairs is often too
costly and challenging, particularly considering their marginal impact on
improving the current model accuracy. Instead, additional or complementary
annotations on the existing input texts in the benchmarks can be preferable as
an efficient way to pay the additional human cost. In this paper, we
investigate task-specific preferences between pairs of input texts as a new
alternative way for such auxiliary data annotation. From 'pair-wise'
comparisons with respect to the task, the auxiliary preference learning enables
the model to learn an additional informative training signal that cannot be
captured with 'instance-wise' task labels. To this end, we propose a novel
multi-task learning framework, called prefer-to-classify (P2C), which can enjoy
the cooperative effect of learning both the given classification task and the
auxiliary preferences. Here, we provide three different ways to collect
preference signals in practice: (a) implicitly extracting from annotation
records (for free, but often unavailable), (b) collecting explicitly from crowd
workers (high paid), or (c) pre-trained large language models such as GPT-3
(low paid). Given existing classification NLP benchmarks, we demonstrate that
the proposed auxiliary preference learning via P2C on them is effective in
improving text classifiers. Our codes are publicly available.

通过进行输入文本间的偏好学习，利用人为标注的 NLP 数据集，提供了用于提高文本分类器的理想补充。