Cultural bias is pervasive in many large language models (LLMs), largely due
to the deficiency of data representative of different cultures. Typically,
cultural datasets and benchmarks are constructed either by extracting subsets
of existing datasets or by aggregating from platforms such as Wikipedia and
social media. However, these approaches are highly dependent on real-world data
and human annotations, making them costly and difficult to scale. Inspired by
cognitive theories on social communication, this paper introduces CulturePark,
an LLM-powered multi-agent communication framework for cultural data
collection. CulturePark simulates cross-cultural human communication with
LLM-based agents playing roles in different cultures. It generates high-quality
cross-cultural dialogues encapsulating human beliefs, norms, and customs. Using
CulturePark, we generated 41,000 cultural samples to fine-tune eight
culture-specific LLMs. We evaluated these models across three downstream tasks:
content moderation, cultural alignment, and cultural education. Results show
that for content moderation, our GPT-3.5-based models either match or
outperform GPT-4 on datasets. Regarding cultural alignment, our models surpass
GPT-4 on Hofstede's VSM 13 framework. Furthermore, for cultural education of
human participants, our models demonstrate superior outcomes in both learning
efficacy and user experience compared to GPT-4. CulturePark proves an important
step in addressing cultural bias and advancing the democratization of AI,
highlighting the critical role of culturally inclusive data in model training.

通过 CulturePark，我们生成了 41,000 个文化样本，用于优化八个特定文化的 LLM。我们在内容审查、文化协调和文化教育三个下游任务中评估了这些模型，结果表明在内容审查方面，基于 GPT-3.5 的模型要么与 GPT-4 相匹配，要么胜过它。在文化协调方面，我们的模型优于 GPT-4 的 Hofstede 的 VSM 13 框架。此外，针对人类参与者的文化教育，我们的模型在学习效果和用户体验方面与 GPT-4 相比展现了卓越的结果。CulturePark 是解决文化偏见和推动人工智能民主化的重要进展，突显了文化包容性数据在模型训练中的关键作用。

CulturePark: 大型语言模型中跨文化理解的提升

CulturePark: Boosting Cross-cultural Understanding in Large Language  Models

Neural Machine Translation (NMT) has shown drastic improvement in its quality
when translating clean input, such as text from the news domain. However,
existing studies suggest that NMT still struggles with certain kinds of input
with considerable noise, such as User-Generated Contents (UGC) on the Internet.
To make better use of NMT for cross-cultural communication, one of the most
promising directions is to develop a model that correctly handles these
expressions. Though its importance has been recognized, it is still not clear
as to what creates the great gap in performance between the translation of
clean input and that of UGC. To answer the question, we present a new dataset,
PheMT, for evaluating the robustness of MT systems against specific linguistic
phenomena in Japanese-English translation. Our experiments with the created
dataset revealed that not only our in-house models but even widely used
off-the-shelf systems are greatly disturbed by the presence of certain
phenomena.

研究使用新数据集 PheMT，评估神经机器翻译系统对于特定日语 - 英语语言现象的鲁棒性，结果显示现有模型在用户生成内容的翻译质量表现不佳。