Conducting experiments with diverse participants in their native languages
can uncover insights into culture, cognition, and language that may not be
revealed otherwise. However, conducting these experiments online makes it
difficult to validate self-reported language proficiency. Furthermore, existing
proficiency tests are small and cover only a few languages. We present an
automated pipeline to generate vocabulary tests using text from Wikipedia. Our
pipeline samples rare nouns and creates pseudowords with the same low-level
statistics. Six behavioral experiments (N=236) in six countries and eight
languages show that (a) our test can distinguish between native speakers of
closely related languages, (b) the test is reliable ($r=0.82$), and (c)
performance strongly correlates with existing tests (LexTale) and self-reports.
We further show that test accuracy is negatively correlated with the linguistic
distance between the tested and the native language. Our test, available in
eight languages, can easily be extended to other languages.

通过 Wikipedia 生成词汇测试，评估参与者的母语能力，可以在不同语言文化背景下了解文化、认知和语言的视角，通过六项行为实验测试结果表明，这个测试可以区分母语相近的语言，测试结果与现有测试成绩和个人报告结果有很强的相关性。

环游世界 60 词：用于在线研究的生成性词汇测试

Around the world in 60 words: A generative vocabulary test for online research

We first adopt a human-guided machine learning framework to develop a new
dataset for hyperpartisan news title detection with 2,200 manually labeled and
1.8 million machine-labeled titles that were posted from 2014 to the present by
nine representative media organizations across three media bias groups - Left,
Central, and Right in an active learning manner. The fine-tuned
transformer-based language model achieves an overall accuracy of 0.84 and an F1
score of 0.78 on an external validation set. Next, we conduct a computational
analysis to quantify the extent and dynamics of partisanship in news titles.
While some aspects are as expected, our study reveals new or nuanced
differences between the three media groups. We find that overall the Right
media tends to use proportionally more hyperpartisan titles. Roughly around the
2016 Presidential Election, the proportions of hyperpartisan titles increased
in all media bias groups where the relative increase in the proportion of
hyperpartisan titles of the Left media was the most. We identify three major
topics including foreign issues, political systems, and societal issues that
are suggestive of hyperpartisanship in news titles using logistic regression
models and the Shapley values. Through an analysis of the topic distribution,
we find that societal issues gradually receive more attention from all media
groups. We further apply a lexicon-based language analysis tool to the titles
of each topic and quantify the linguistic distance between any pairs of the
three media groups. Three distinct patterns are discovered. The Left media is
linguistically more different from Central and Right in terms of foreign
issues. The linguistic distance between the three media groups becomes smaller
over recent years. In addition, a seasonal pattern where linguistic difference
is associated with elections is observed for societal issues.

本文采取人工辅助机器学习框架，针对具有代表性的 3 个主流媒体倾向组（左，中，右）在 2014 年至今发布的 2.2 万份手动标记和 180 万份机器标记的新闻标题进行了超党派新闻标题检测的数据集开发，并在外部验证集上取得了 0.84 的准确度和 0.78 的 F1 得分；针对新闻标题中的党派性进行了计算分析，发现右翼媒体倾向于使用比例更高的超党派标题；通过逻辑回归模型和 Shapley 值确定了几个主题，并应用基于词库的语言分析工具对每个主题的标题进行了分析，发现左派、中派和右派媒体在外交、政治和社会问题方面的语言差异逐渐减小。