Productive interactions between diverse users and language technologies
require outputs from the latter to be culturally relevant and sensitive. Prior
works have evaluated models' knowledge of cultural norms, values, and
artifacts, without considering how this knowledge manifests in downstream
applications. In this work, we focus on extrinsic evaluation of cultural
competence in two text generation tasks, open-ended question answering and
story generation. We quantitatively and qualitatively evaluate model outputs
when an explicit cue of culture, specifically nationality, is perturbed in the
prompts. Although we find that model outputs do vary when varying nationalities
and feature culturally relevant words, we also find weak correlations between
text similarity of outputs for different countries and the cultural values of
these countries. Finally, we discuss important considerations in designing
comprehensive evaluation of cultural competence in user-facing tasks.

本文研究了多样化用户和语言技术之间的高效交互所需的文化相关性和敏感性，以及在两个文本生成任务中文化能力的外部评估，最终讨论了在用户面向任务中设计全面评估文化能力时的重要考虑因素。

大型语言模型中文化能力的外在评估

Extrinsic Evaluation of Cultural Competence in Large Language Models

Fake news detection plays a crucial role in protecting social media users and
maintaining a healthy news ecosystem. Among existing works, comment-based fake
news detection methods are empirically shown as promising because comments
could reflect users' opinions, stances, and emotions and deepen models'
understanding of fake news. Unfortunately, due to exposure bias and users'
different willingness to comment, it is not easy to obtain diverse comments in
reality, especially for early detection scenarios. Without obtaining the
comments from the ``silent'' users, the perceived opinions may be incomplete,
subsequently affecting news veracity judgment. In this paper, we explore the
possibility of finding an alternative source of comments to guarantee the
availability of diverse comments, especially those from silent users.
Specifically, we propose to adopt large language models (LLMs) as a user
simulator and comment generator, and design GenFEND, a generated
feedback-enhanced detection framework, which generates comments by prompting
LLMs with diverse user profiles and aggregating generated comments from
multiple subpopulation groups. Experiments demonstrate the effectiveness of
GenFEND and further analysis shows that the generated comments cover more
diverse users and could even be more effective than actual comments.

通过采用大型语言模型作为用户模拟器和评论生成器，我们提出了 GenFEND，一种生成的反馈增强检测框架，通过为多个子人群组提示大型语言模型并聚合生成的评论，以做到多样化评论的可用性，尤其是来自沉默用户的评论。实验证明了 GenFEND 的有效性，并进一步分析表明，生成的评论涵盖了更多多样化的用户，并且甚至比实际评论更有效。

让沉默发声：利用大型语言模型生成评论增强假新闻检测

Let Silence Speak: Enhancing Fake News Detection with Generated Comments  from Large Language Models

We investigate non-collaborative dialogue agents that must engage in tailored
strategic planning for diverse users to secure a favorable agreement. This
poses challenges for existing dialogue agents due to two main reasons: their
inability to integrate user-specific characteristics into their strategic
planning and their training paradigm's failure to produce strategic planners
that can generalize to diverse users. To address these challenges, we propose
TRIP to enhance the capability in tailored strategic planning, incorporating a
user-aware strategic planning module and a population-based training paradigm.
Through experiments on benchmark non-collaborative dialogue tasks, we
demonstrate the effectiveness of TRIP in catering to diverse users.

提出了 TRIP 用于增强定制战略规划的能力，通过引入用户感知的战略规划模块和基于人口统计的训练范式，解决了现有对话代理面临的两大挑战：无法将用户特定特征纳入战略规划以及无法对多样化用户进行泛化训练。通过在标准的非合作对话任务上进行实验，证明了 TRIP 在迎合多样化用户方面的有效性。