The integration of Large Language Models (LLMs) into various global cultures
fundamentally presents a cultural challenge: LLMs must navigate interactions,
respect social norms, and avoid transgressing cultural boundaries. However, it
is still unclear if LLMs can adapt their outputs to diverse cultural norms. Our
study focuses on this aspect. We introduce NormAd, a novel dataset, which
includes 2.6k stories that represent social and cultural norms from 75
countries, to assess the ability of LLMs to adapt to different granular levels
of socio-cultural contexts such as the country of origin, its associated
cultural values, and prevalent social norms. Our study reveals that LLMs
struggle with cultural reasoning across all contextual granularities, showing
stronger adaptability to English-centric cultures over those from the Global
South. Even with explicit social norms, the top-performing model,
Mistral-7b-Instruct, achieves only 81.8\% accuracy, lagging behind the 95.6\%
achieved by humans. Evaluation on NormAd further reveals that LLMs struggle to
adapt to stories involving gift-giving across cultures. Due to inherent
agreement or sycophancy biases, LLMs find it considerably easier to assess the
social acceptability of stories that adhere to cultural norms than those that
deviate from them. Our benchmark measures the cultural adaptability (or lack
thereof) of LLMs, emphasizing the potential to make these technologies more
equitable and useful for global audiences.

我们的研究关注大型语言模型（LLMs）适应不同社会文化规范的能力，通过介绍 NormAd 数据集评估 LLMs 在不同社会文化情境中适应性的表现。研究发现 LLMs 在文化推理上存在困难，尤其对于来自南方全球的文化相对于以英语为中心的文化更难适应。LLMs 在评估跨文化礼物给予的故事时表现较差，对于符合文化规范的故事比违背规范的故事更容易评估社会可接受性。我们的基准测试衡量了 LLMs 的文化适应性，并强调了使这些技术对全球受众更加公平和有用的潜力。