Exploring and understanding language data is a fundamental stage in all areas
dealing with human language. It allows NLP practitioners to uncover quality
concerns and harmful biases in data before training, and helps linguists and
social scientists to gain insight into language use and human behavior. Yet,
there is currently a lack of a unified, customizable tool to seamlessly inspect
and visualize language variation and bias across multiple variables, language
units, and diverse metrics that go beyond descriptive statistics. In this
paper, we introduce Variationist, a highly-modular, extensible, and
task-agnostic tool that fills this gap. Variationist handles at once a
potentially unlimited combination of variable types and semantics across
diversity and association metrics with regards to the language unit of choice,
and orchestrates the creation of up to five-dimensional interactive charts for
over 30 variable type-semantics combinations. Through our case studies on
computational dialectology, human label variation, and text generation, we show
how Variationist enables researchers from different disciplines to effortlessly
answer specific research questions or unveil undesired associations in language
data. A Python library, code, documentation, and tutorials are made publicly
available to the research community.

探索和理解语言数据是处理人类语言的所有领域的基础阶段，Variationist 是一个高度模块化、可扩展且与任务无关的工具，通过多个变量、语言单位和多样的度量方式进行无缝检查和可视化语言变异和偏见，实现了 30 多种变量类型 - 语义组合的五维交互式图表的创建，帮助研究人员轻松回答特定的研究问题或揭示语言数据中不希望的关联。

变异主义：探索书面语言数据中的多元化变异和偏见

Variationist: Exploring Multifaceted Variation and Bias in Written  Language Data

A myriad of explainability methods have been proposed in recent years, but
there is little consensus on how to evaluate them. While automatic metrics
allow for quick benchmarking, it isn't clear how such metrics reflect human
interaction with explanations. Human evaluation is of paramount importance, but
previous protocols fail to account for belief biases affecting human
performance, which may lead to misleading conclusions. We provide an overview
of belief bias, its role in human evaluation, and ideas for NLP practitioners
on how to account for it. For two experimental paradigms, we present a case
study of gradient-based explainability introducing simple ways to account for
humans' prior beliefs: models of varying quality and adversarial examples. We
show that conclusions about the highest performing methods change when
introducing such controls, pointing to the importance of accounting for belief
bias in evaluation.

该研究介绍了解释性方法的评估问题，说明了评估中信仰偏见的重要性并提出了 NLP 从业者应该如何解决这一问题，同时通过基于梯度的解释性案例研究得到，在评估中考虑个体先验信仰的重要性。