Conversational recommendation (ConvRec) systems must understand rich and
diverse natural language (NL) expressions of user preferences and intents,
often communicated in an indirect manner (e.g., "I'm watching my weight"). Such
complex utterances make retrieving relevant items challenging, especially if
only using often incomplete or out-of-date metadata. Fortunately, many domains
feature rich item reviews that cover standard metadata categories and offer
complex opinions that might match a user's interests (e.g., "classy joint for a
date"). However, only recently have large language models (LLMs) let us unlock
the commonsense connections between user preference utterances and complex
language in user-generated reviews. Further, LLMs enable novel paradigms for
semi-structured dialogue state tracking, complex intent and preference
understanding, and generating recommendations, explanations, and question
answers. We thus introduce a novel technology RA-Rec, a Retrieval-Augmented,
LLM-driven dialogue state tracking system for ConvRec, showcased with a video,
open source GitHub repository, and interactive Google Colab notebook.

对话式推荐系统必须理解用户偏好和意图的丰富多样的自然语言表达，而这些表达通常以间接方式传达（例如：“我在控制体重”）。这些复杂的话语使得检索相关物品变得具有挑战性，尤其是当只使用通常不完整或过时的元数据时。幸运的是，许多领域都有丰富的商品评价，这些评价涵盖标准的元数据类别并提供与用户兴趣相匹配的复杂意见（例如：“适合约会的高档场所”）。然而，直到最近，大尺度语言模型让我们能够揭示用户偏好表达和用户生成评价之间的常识联系。进一步而言，大尺度语言模型还能够实现半结构化对话状态跟踪、复杂意图和偏好理解，以及生成推荐、解释和问题回答等新颖范式。因此，我们引入了一种名为 RA-Rec 的新技术，一种以大尺度语言模型为驱动的检索增强型对话状态跟踪系统，用视频、开源 GitHub 存储库和可交互的 Google Colab 笔记本展示了这项技术。

基于提示的半结构化自然语言状态追踪的检索增强型对话推荐

Retrieval-Augmented Conversational Recommendation with Prompt-based  Semi-Structured Natural Language State Tracking

Widely deployed large language models (LLMs) can produce convincing yet
incorrect outputs, potentially misleading users who may rely on them as if they
were correct. To reduce such overreliance, there have been calls for LLMs to
communicate their uncertainty to end users. However, there has been little
empirical work examining how users perceive and act upon LLMs' expressions of
uncertainty. We explore this question through a large-scale, pre-registered,
human-subject experiment (N=404) in which participants answer medical questions
with or without access to responses from a fictional LLM-infused search engine.
Using both behavioral and self-reported measures, we examine how different
natural language expressions of uncertainty impact participants' reliance,
trust, and overall task performance. We find that first-person expressions
(e.g., "I'm not sure, but...") decrease participants' confidence in the system
and tendency to agree with the system's answers, while increasing participants'
accuracy. An exploratory analysis suggests that this increase can be attributed
to reduced (but not fully eliminated) overreliance on incorrect answers. While
we observe similar effects for uncertainty expressed from a general perspective
(e.g., "It's not clear, but..."), these effects are weaker and not
statistically significant. Our findings suggest that using natural language
expressions of uncertainty may be an effective approach for reducing
overreliance on LLMs, but that the precise language used matters. This
highlights the importance of user testing before deploying LLMs at scale.

广泛应用的大型语言模型（LLM）能够生成具有说服力但不正确的结果，存在误导用户的风险。为减少过度依赖，有呼吁要求 LLM 向最终用户传达其不确定性。然而，有关用户如何感知和行动 LLM 的不确定表达的实证研究很少。通过大规模的、预先注册的人体实验（N=404），我们探索了这个问题，参与者回答医学问题，带或不带来自一个虚构的 LLM 增强搜索引擎的回复。通过行为和自我报告的测量，我们研究了不同的自然语言不确定表达方式对参与者信赖、信任和任务整体表现的影响。我们发现第一人称表达（如 “我不确定，但是...”）降低了参与者对系统的信心和同意系统答案的倾向，同时提高了参与者的准确性。初步分析表明，这种增加可以归因于对错误答案的减少（但并非完全消除）的过度依赖。虽然我们观察到了不确定性从一般角度表达的类似效果（如 “不清楚，但是...”），但这些效果较弱且不具有统计学意义。我们的发现表明，使用自然语言的不确定表达可能是减少对 LLM 过度依赖的有效方法，但使用的确切语言很重要。这凸显了在部署大规模 LLM 之前进行用户测试的重要性。

大型语言模型的不确定表达对用户依赖和信任的影响研究

"I'm Not Sure, But...": Examining the Impact of Large Language Models'  Uncertainty Expression on User Reliance and Trust

Referring video segmentation relies on natural language expressions to
identify and segment objects, often emphasizing motion clues. Previous works
treat a sentence as a whole and directly perform identification at the
video-level, mixing up static image-level cues with temporal motion cues.
However, image-level features cannot well comprehend motion cues in sentences,
and static cues are not crucial for temporal perception. In fact, static cues
can sometimes interfere with temporal perception by overshadowing motion cues.
In this work, we propose to decouple video-level referring expression
understanding into static and motion perception, with a specific emphasis on
enhancing temporal comprehension. Firstly, we introduce an
expression-decoupling module to make static cues and motion cues perform their
distinct role, alleviating the issue of sentence embeddings overlooking motion
cues. Secondly, we propose a hierarchical motion perception module to capture
temporal information effectively across varying timescales. Furthermore, we
employ contrastive learning to distinguish the motions of visually similar
objects. These contributions yield state-of-the-art performance across five
datasets, including a remarkable $\textbf{9.2%}$ $\mathcal{J\&F}$ improvement
on the challenging $\textbf{MeViS}$ dataset. Code is available at
this https URL

视频级别参照表达理解的静态与运动感知的解耦以及对时间感知的强化，并采用对比学习来区分视觉上相似的对象的运动，取得了在五个数据集上的最先进性能，并在具有挑战性的 MeViS 数据集上有了显著的 9.2% 的 J&F 改进。