Effective interlocutors account for the uncertain goals, beliefs, and
emotions of others. But even the best human conversationalist cannot perfectly
anticipate the trajectory of a dialogue. How well can language models represent
inherent uncertainty in conversations? We propose FortUne Dial, an expansion of
the long-standing "conversation forecasting" task: instead of just accuracy,
evaluation is conducted with uncertainty-aware metrics, effectively enabling
abstention on individual instances. We study two ways in which language models
potentially represent outcome uncertainty (internally, using scores and
directly, using tokens) and propose fine-tuning strategies to improve
calibration of both representations. Experiments on eight difficult negotiation
corpora demonstrate that our proposed fine-tuning strategies (a traditional
supervision strategy and an off-policy reinforcement learning strategy) can
calibrate smaller open-source models to compete with pre-trained models 10x
their size.

有效的对话者必须考虑他人的不确定目标、信念和情绪，但是即使是最好的人类对话者也无法完美地预测对话的轨迹。我们提出了 FortUne Dial，对长期存在的 “对话预测” 任务进行了扩展：不仅评估准确性，还使用了具有不确定性意识的度量标准，有效地使个别实例中可以放弃预测。我们研究了语言模型在代表对话结果不确定性方面的两种方式（内部使用分数，直接使用标记），并提出了改进这两种表示的校准的微调策略。在八个困难的谈判语料库上的实验证明，我们提出的微调策略（传统的监督策略和离线策略增强学习策略）能够使规模较小的开源模型与其体积为 10 倍的预训练模型竞争。

使用大型语言模型预测对话中的不确定性：成交与否（或谁知道）？

Deal, or no deal (or who knows)? Forecasting Uncertainty in  Conversations using Large Language Models

Investigating cooperativity of interlocutors is central in studying
pragmatics of dialogue. Models of conversation that only assume cooperative
agents fail to explain the dynamics of strategic conversations. Thus, we
investigate the ability of agents to identify non-cooperative interlocutors
while completing a concurrent visual-dialogue task. Within this novel setting,
we study the optimality of communication strategies for achieving this
multi-task objective. We use the tools of learning theory to develop a
theoretical model for identifying non-cooperative interlocutors and apply this
theory to analyze different communication strategies. We also introduce a
corpus of non-cooperative conversations about images in the GuessWhat?! dataset
proposed by De Vries et al. (2017). We use reinforcement learning to implement
multiple communication strategies in this context and find empirical results
validate our theory.

本文研究了在完成一个同时进行的视觉 - 对话任务时，如何识别非合作对话者，使用学习理论的工具开发了理论模型，提出了多种沟通策略并应用强化学习来实现，通过对 GuessWhat?! 数据集的非合作对话语料库的实验结果验证了理论模型的有效性。

建模非合作式对话：理论和实证研究

Modeling Non-Cooperative Dialogue: Theoretical and Empirical Insights

Recently, various response generation models for two-party conversations have
achieved impressive improvements, but less effort has been paid to multi-party
conversations (MPCs) which are more practical and complicated. Compared with a
two-party conversation where a dialogue context is a sequence of utterances,
building a response generation model for MPCs is more challenging, since there
exist complicated context structures and the generated responses heavily rely
on both interlocutors (i.e., speaker and addressee) and history utterances. To
address these challenges, we present HeterMPC, a heterogeneous graph-based
neural network for response generation in MPCs which models the semantics of
utterances and interlocutors simultaneously with two types of nodes in a graph.
Besides, we also design six types of meta relations with
node-edge-type-dependent parameters to characterize the heterogeneous
interactions within the graph. Through multi-hop updating, HeterMPC can
adequately utilize the structural knowledge of conversations for response
generation. Experimental results on the Ubuntu Internet Relay Chat (IRC)
channel benchmark show that HeterMPC outperforms various baseline models for
response generation in MPCs.

提出了一种基于异构图神经网络的多方会话响应生成模型，通过建模话语和交互者的语义信息及设计不同类型的元关系来处理多方对话中的结构信息，实验证明其在 Ubuntu Internet Relay Chat（IRC）通道基准测试上优于其他基线模型。