We present a survey of 39 recent papers that aim to study cultural
representation and inclusion in large language models. We observe that none of
the studies define "culture," which is a complex, multifaceted concept;
instead, they probe the models on some specially designed datasets which
represent certain aspects of "culture." We call these aspects the proxies of
cultures, and organize them across three dimensions of demographic, semantic
and linguistic-cultural interaction proxies. We also categorize the probing
methods employed. Our analysis indicates that only certain aspects of
"culture," such as values and objectives, have been studied, leaving several
other interesting and important facets, especially the multitude of semantic
domains (Thompson et al., 2020) and aboutness (Hershcovich et al., 2022),
unexplored. Two other crucial gaps are the lack of robustness and situatedness
of the current methods. Based on these observations, we provide several
recommendations for a holistic and practically useful research agenda for
furthering cultural inclusion in LLMs and LLM-based applications.

对研究大型语言模型中文化表示和包容性的 39 篇最新论文进行概述，发现这些研究中没有明确定义 “文化”，而是通过一些特殊设计的数据集来探索模型，这些数据集代表了文化的某些方面；我们称这些方面为文化的代理，按人口统计学、语义和语言文化交互代理三个维度将其组织起来，并对使用的探测方法进行分类。分析表明，只有文化的某些方面，如价值观和目标，得到了研究，其他一些有趣且重要的方面，尤其是语义领域的多样性和主题向度，尚未被探索。另外，目前的方法缺乏鲁棒性和情境性。基于这些观察，提供了关于进一步促进 LLM 和基于 LLM 的应用中文化包容的全面可行的研究议程的几点建议。

测量和建模 LLMs 中的 “文化”: 一项调查

Towards Measuring and Modeling "Culture" in LLMs: A Survey

Interpretability methods in NLP aim to provide insights into the semantics
underlying specific system architectures. Focusing on word embeddings, we
present a supervised-learning method that, for a given domain (e.g., sports,
professions), identifies a subset of model features that strongly improve
prediction of human similarity judgments. We show this method keeps only 20-40%
of the original embeddings, for 8 independent semantic domains, and that it
retains different feature sets across domains. We then present two approaches
for interpreting the semantics of the retained features. The first obtains the
scores of the domain words (co-hyponyms) on the first principal component of
the retained embeddings, and extracts terms whose co-occurrence with the
co-hyponyms tracks these scores' profile. This analysis reveals that humans
differentiate e.g. sports based on how gender-inclusive and international they
are. The second approach uses the retained sets as variables in a probing task
that predicts values along 65 semantically annotated dimensions for a dataset
of 535 words. The features retained for professions are best at predicting
cognitive, emotional and social dimensions, whereas features retained for
fruits or vegetables best predict the gustation (taste) dimension. We discuss
implications for alignment between AI systems and human knowledge.

通过一种有监督学习方法，针对特定领域（例如运动、职业），识别出一组强烈改善人类相似度判断预测的模型特征子集，并通过两种方法解释所保留特征的语义，从而揭示了人类如何根据性别包容性和国际性来区分运动类别，以及不同领域中特征对不同语义维度的预测能力。对于职业特征而言，最能预测认知、情感和社会维度，而对于水果或蔬菜特征而言，则最能预测味觉维度。讨论了对于人工智能系统与人类知识之间的对齐的意义。