Large language models (LLMs) have demonstrated substantial commonsense
understanding through numerous benchmark evaluations. However, their
understanding of cultural commonsense remains largely unexamined. In this
paper, we conduct a comprehensive examination of the capabilities and
limitations of several state-of-the-art LLMs in the context of cultural
commonsense tasks. Using several general and cultural commonsense benchmarks,
we find that (1) LLMs have a significant discrepancy in performance when tested
on culture-specific commonsense knowledge for different cultures; (2) LLMs'
general commonsense capability is affected by cultural context; and (3) The
language used to query the LLMs can impact their performance on
cultural-related tasks. Our study points to the inherent bias in the cultural
understanding of LLMs and provides insights that can help develop culturally
aware language models.

通过对多个最先进的大型语言模型的文化常识任务的能力和限制进行全面检验，我们发现大型语言模型在文化特定的常识知识上的表现存在显著差异，其通用常识能力受到文化环境的影响，并且提出查询大型语言模型所使用的语言会影响其在与文化相关的任务上的表现，我们的研究指出了大型语言模型在文化理解方面的固有偏见，并提供了帮助开发具备文化意识的语言模型的洞见。