Exploring and understanding language data is a fundamental stage in all areas
dealing with human language. It allows NLP practitioners to uncover quality
concerns and harmful biases in data before training, and helps linguists and
social scientists to gain insight into language use and human behavior. Yet,
there is currently a lack of a unified, customizable tool to seamlessly inspect
and visualize language variation and bias across multiple variables, language
units, and diverse metrics that go beyond descriptive statistics. In this
paper, we introduce Variationist, a highly-modular, extensible, and
task-agnostic tool that fills this gap. Variationist handles at once a
potentially unlimited combination of variable types and semantics across
diversity and association metrics with regards to the language unit of choice,
and orchestrates the creation of up to five-dimensional interactive charts for
over 30 variable type-semantics combinations. Through our case studies on
computational dialectology, human label variation, and text generation, we show
how Variationist enables researchers from different disciplines to effortlessly
answer specific research questions or unveil undesired associations in language
data. A Python library, code, documentation, and tutorials are made publicly
available to the research community.

探索和理解语言数据是处理人类语言的所有领域的基础阶段，Variationist 是一个高度模块化、可扩展且与任务无关的工具，通过多个变量、语言单位和多样的度量方式进行无缝检查和可视化语言变异和偏见，实现了 30 多种变量类型 - 语义组合的五维交互式图表的创建，帮助研究人员轻松回答特定的研究问题或揭示语言数据中不希望的关联。

变异主义：探索书面语言数据中的多元化变异和偏见

Variationist: Exploring Multifaceted Variation and Bias in Written  Language Data

Document scaling has been a key component in text-as-data applications for
social scientists and a major field of interest for political researchers, who
aim at uncovering differences between speakers or parties with the help of
different probabilistic and non-probabilistic approaches. Yet, most of these
techniques are either built upon the agnostically bag-of-word hypothesis or use
prior information borrowed from external sources that might embed the results
with a significant bias. If the corpus has long been considered as a collection
of documents, it can also be seen as a dense network of connected words whose
structure could be clustered to differentiate independent groups of words,
based on their co-occurrences in documents, known as communities. This paper
introduces CommunityFish as an augmented version of Wordfish based on a
hierarchical clustering, namely the Louvain algorithm, on the word space to
yield communities as semantic and independent n-grams emerging from the corpus
and use them as an input to Wordfish method, instead of considering the word
space. This strategy emphasizes the interpretability of the results, since
communities have a non-overlapping structure, hence a crucial informative power
in discriminating parties or speakers, in addition to allowing a faster
execution of the Poisson scaling model. Aside from yielding communities,
assumed to be subtopic proxies, the application of this technique outperforms
the classic Wordfish model by highlighting historical developments in the U.S.
State of the Union addresses and was found to replicate the prevailing
political stance in Germany when using the corpus of parties' legislative
manifestos.

该研究介绍了 CommunityFish，它是 Wordfish 的一个增强版本，使用层次聚类方法将语料库中的词语空间聚类为语义和独立的 n-grams 社群，并将其作为 Wordfish 方法的输入，以强调结果的可解释性和提高执行速度。通过突出美国国情咨文的历史发展，并使用政党立法宣言的语料库，该技术不仅生成了社群，还能够复制德国的主导政治立场。