Research on dialogue constructiveness assessment focuses on (i) analysing
conversational factors that influence individuals to take specific actions, win
debates, change their perspectives or broaden their open-mindedness and (ii)
predicting constructive outcomes following dialogues for such use cases. These
objectives can be achieved by training either interpretable feature-based
models (which often involve costly human annotations) or neural models such as
pre-trained language models (which have empirically shown higher task accuracy
but lack interpretability). We propose a novel LLM feature-based framework that
combines the strengths of feature-based and neural approaches while mitigating
their downsides, in assessing dialogue constructiveness. The framework first
defines a set of dataset-independent and interpretable linguistic features,
which can be extracted by both prompting an LLM and simple heuristics. Such
features are then used to train LLM feature-based models. We apply this
framework to three datasets of dialogue constructiveness and find that our LLM
feature-based models significantly outperform standard feature-based models and
neural models, and tend to learn more robust prediction rules instead of
relying on superficial shortcuts (as seen with neural models). Further, we
demonstrate that interpreting these LLM feature-based models can yield valuable
insights into what makes a dialogue constructive.

我们提出了一种新颖的 LLM 基于特征的框架，将基于特征的方法和神经网络方法的优势结合起来，评估对话的建设性。这个框架首先定义了一组与数据集无关且可解释的语言特征，然后利用这些特征训练 LLM 基于特征的模型，从而比传统基于特征的模型和神经模型表现更好，学习更强大的预测规则，而不是依赖表面的捷径。此外，我们证明解释这些 LLM 基于特征的模型可以获得有价值的对话建设性见解。