The deployment of large language models (LLMs) raises concerns regarding
their cultural misalignment and potential ramifications on individuals from
various cultural norms. Existing work investigated political and social biases
and public opinions rather than their cultural values. To address this
limitation, the proposed Cultural Alignment Test (CAT) quantifies cultural
alignment using Hofstede's cultural dimension framework, which offers an
explanatory cross-cultural comparison through the latent variable analysis. We
apply our approach to assess the cultural values embedded in state-of-the-art
LLMs, such as: ChatGPT and Bard, across diverse cultures of countries: United
States (US), Saudi Arabia, China, and Slovakia, using different prompting
styles and hyperparameter settings. Our results not only quantify cultural
alignment of LLMs with certain countries, but also reveal the difference
between LLMs in explanatory cultural dimensions. While all LLMs did not provide
satisfactory results in understanding cultural values, GPT-4 exhibited the
highest CAT score for the cultural values of the US.

使用 Hofstede 的文化维度框架，通过潜变量分析提供解释性跨文化比较，文中提出了 Cultural Alignment Test（CAT）以解决大型语言模型（LLMs）的文化不匹配问题，并使用不同的提示样式和超参数设置评估了 ChatGPT 和 Bard 等 LLMs 嵌入在美国、沙特阿拉伯、中国和斯洛伐克等不同文化之中的文化价值。结果不仅量化了 LLMs 与特定国家的文化一致性，还揭示了 LLMs 在解释性文化维度方面的差异。其中 GPT-4 在理解美国文化价值方面得分最高。