Rapport is known as a conversational aspect focusing on relationship
building, which influences outcomes in collaborative tasks. This study aims to
establish human-agent rapport through small talk by using a rapport-building
strategy. We implemented this strategy for the virtual agents based on dialogue
strategies by prompting a large language model (LLM). In particular, we
utilized two dialogue strategies-predefined sequence and free-form-to guide the
dialogue generation framework. We conducted analyses based on human
evaluations, examining correlations between total turn, utterance characters,
rapport score, and user experience variables: naturalness, satisfaction,
interest, engagement, and usability. We investigated correlations between
rapport score and naturalness, satisfaction, engagement, and conversation flow.
Our experimental results also indicated that using free-form to prompt the
rapport-building strategy performed the best in subjective scores.

通过使用一种建立关系的策略，本研究旨在通过闲聊来建立人机关系，并通过大型语言模型（LLM）为虚拟代理实现这种策略。通过预定义的顺序和自由形式的对话策略，分析了人类评估、对话得分、自然度、满意度、兴趣、参与度和可用性之间的相关性。实验结果表明，使用自由形式来促进建立关系的策略在主观评分方面表现最佳。

基于沟通的虚拟代理：在初次见面时改进用户体验的沟通策略

Rapport-Driven Virtual Agent: Rapport Building Dialogue Strategy for  Improving User Experience at First Meeting

As Large Language Models (LLMs) are predictive models building their response
based on the words in the prompts, there is a risk that small talk and
irrelevant information may alter the response and the suggestion given.
Therefore, this study aims to investigate the impact of medical data mixed with
small talk on the accuracy of medical advice provided by ChatGPT. USMLE step 3
questions were used as a model for relevant medical data. We use both multiple
choice and open ended questions. We gathered small talk sentences from human
participants using the Mechanical Turk platform. Both sets of USLME questions
were arranged in a pattern where each sentence from the original questions was
followed by a small talk sentence. ChatGPT 3.5 and 4 were asked to answer both
sets of questions with and without the small talk sentences. A board-certified
physician analyzed the answers by ChatGPT and compared them to the formal
correct answer. The analysis results demonstrate that the ability of
ChatGPT-3.5 to answer correctly was impaired when small talk was added to
medical data for multiple-choice questions (72.1\% vs. 68.9\%) and open
questions (61.5\% vs. 44.3\%; p=0.01), respectively. In contrast, small talk
phrases did not impair ChatGPT-4 ability in both types of questions (83.6\% and
66.2\%, respectively). According to these results, ChatGPT-4 seems more
accurate than the earlier 3.5 version, and it appears that small talk does not
impair its capability to provide medical recommendations. Our results are an
important first step in understanding the potential and limitations of
utilizing ChatGPT and other LLMs for physician-patient interactions, which
include casual conversations.

研究试图调查医学数据与闲聊混合对 ChatGPT 提供的医疗建议准确性的影响，结果显示 ChatGPT-4 的准确性比 3.5 版本更高，并且似乎闲聊不会影响其提供医疗建议的能力，这对于理解利用 ChatGPT 和其他 LLM 进行医患互动的潜力和限制是重要的第一步。

ChatGPT-3.5 和 GPT-4 在美国医疗执照考试中的表现 —— 带与不带干扰的比较

Performance of ChatGPT-3.5 and GPT-4 on the United States Medical  Licensing Examination With and Without Distractions

Open-domain chatbots are supposed to converse freely with humans without
being restricted to a topic, task or domain. However, the boundaries and/or
contents of open-domain conversations are not clear. To clarify the boundaries
of "openness", we conduct two studies: First, we classify the types of "speech
events" encountered in a chatbot evaluation data set (i.e., Meena by Google)
and find that these conversations mainly cover the "small talk" category and
exclude the other speech event categories encountered in real life human-human
communication. Second, we conduct a small-scale pilot study to generate online
conversations covering a wider range of speech event categories between two
humans vs. a human and a state-of-the-art chatbot (i.e., Blender by Facebook).
A human evaluation of these generated conversations indicates a preference for
human-human conversations, since the human-chatbot conversations lack coherence
in most speech event categories. Based on these results, we suggest (a) using
the term "small talk" instead of "open-domain" for the current chatbots which
are not that "open" in terms of conversational abilities yet, and (b) revising
the evaluation methods to test the chatbot conversations against other speech
events.

通过两项研究，本文阐述了当前的 Open-domain chatbots 的会话范围的局限性，建议使用 “small talk” 而非 “open-domain” 来描述目前的聊天机器人，同时提出需要改进评估方法以测试聊天机器人与其他语篇的对话。