In this work, we evaluate various existing dialogue relevance metrics, find
strong dependency on the dataset, often with poor correlation with human scores
of relevance, and propose modifications to reduce data requirements and domain
sensitivity while improving correlation. Our proposed metric achieves
state-of-the-art performance on the HUMOD dataset while reducing measured
sensitivity to dataset by 37%-66%. We achieve this without fine-tuning a
pretrained language model, and using only 3,750 unannotated human dialogues and
a single negative example. Despite these limitations, we demonstrate
competitive performance on four datasets from different domains. Our code,
including our metric and experiments, is open sourced.

评估了现有的对话相关性度量标准，提出了减少数据要求和领域敏感性的改进并在不 fine-tuning 预训练语言模型的情况下，只使用 3750 个未经注释的人类对话和一个负面例子，我们的提议指标实现了 HUMOD dataset 的最新性能，并将数据集敏感性降低了 37％-66％。

对话中的相关性：简单优于复杂的度量标准比较与新度量标准

Relevance in Dialogue: Is Less More? An Empirical Comparison of Existing Metrics, and a Novel Simple Metric

Word embeddings have been widely used in sentiment classification because of
their efficacy for semantic representations of words. Given reviews from
different domains, some existing methods for word embeddings exploit sentiment
information, but they cannot produce domain-sensitive embeddings. On the other
hand, some other existing methods can generate domain-sensitive word
embeddings, but they cannot distinguish words with similar contexts but
opposite sentiment polarity. We propose a new method for learning
domain-sensitive and sentiment-aware embeddings that simultaneously capture the
information of sentiment semantics and domain sensitivity of individual words.
Our method can automatically determine and produce domain-common embeddings and
domain-specific embeddings. The differentiation of domain-common and
domain-specific words enables the advantage of data augmentation of common
semantics from multiple domains and capture the varied semantics of specific
words from different domains at the same time. Experimental results show that
our model provides an effective way to learn domain-sensitive and
sentiment-aware word embeddings which benefit sentiment classification at both
sentence level and lexicon term level.

该文介绍了一种新的方法，通过同时捕获情感语义和单词的域敏感性的信息来学习领域敏感且情感感知的词嵌入向量。实验结果表明，该模型在句子和词库级别上都提供了学习领域敏感和情感感知词嵌入向量的有效方法。