Bias is a disproportionate prejudice in favor of one side against another.
Due to the success of transformer-based Masked Language Models (MLMs) and their
impact on many NLP tasks, a systematic evaluation of bias in these models is
needed more than ever. While many studies have evaluated gender bias in English
MLMs, only a few works have been conducted for the task in other languages.
This paper proposes a multilingual approach to estimate gender bias in MLMs
from 5 languages: Chinese, English, German, Portuguese, and Spanish. Unlike
previous work, our approach does not depend on parallel corpora coupled with
English to detect gender bias in other languages using multilingual lexicons.
Moreover, a novel model-based method is presented to generate sentence pairs
for a more robust analysis of gender bias, compared to the traditional
lexicon-based method. For each language, both the lexicon-based and model-based
methods are applied to create two datasets respectively, which are used to
evaluate gender bias in an MLM specifically trained for that language using one
existing and 3 new scoring metrics. Our results show that the previous approach
is data-sensitive and not stable as it does not remove contextual dependencies
irrelevant to gender. In fact, the results often flip when different scoring
metrics are used on the same dataset, suggesting that gender bias should be
studied on a large dataset using multiple evaluation metrics for best practice.

通过多种评估指标，提出了一种多语言方法来估计基于 Transformer 的遮掩语言模型中的性别偏见。同时，通过与传统的基于词典的方法相比，提出了一种基于模型的方法，以更全面和稳健地分析性别偏见。

多语言遮蔽语言模型中的性别偏见评估

What is Your Favorite Gender, MLM? Gender Bias Evaluation in  Multilingual Masked Language Models

Anaphoric reference is an aspect of language interpretation covering a
variety of types of interpretation beyond the simple case of identity reference
to entities introduced via nominal expressions covered by the traditional
coreference task in its most recent incarnation in ONTONOTES and similar
datasets. One of these cases that go beyond simple coreference is anaphoric
reference to entities that must be added to the discourse model via
accommodation, and in particular split-antecedent references to entities
constructed out of other entities, as in split-antecedent plurals and in some
cases of discourse deixis. Although this type of anaphoric reference is now
annotated in many datasets, systems interpreting such references cannot be
evaluated using the Reference coreference scorer Pradhan et al. (2014). As part
of the work towards a new scorer for anaphoric reference able to evaluate all
aspects of anaphoric interpretation in the coverage of the Universal Anaphora
initiative, we propose in this paper a solution to the technical problem of
generalizing existing metrics for identity anaphora so that they can also be
used to score cases of split-antecedents. This is the first such proposal in
the literature on anaphora or coreference, and has been successfully used to
score both split-antecedent plural references and discourse deixis in the
recent CODI/CRAC anaphora resolution in dialogue shared tasks.

本研究提出第一种将现有的认同指代度量泛化，以便它们也可用于评分分裂先行语情况的解决方案。

使用分裂先行词消解打分核心链

Scoring Coreference Chains with Split-Antecedent Anaphors

Autonomous and semi-autonomous vehicles' perception algorithms can encounter
situations with erroneous object detection, such as misclassification of
objects on the road, which can lead to safety violations and potentially fatal
consequences. While there has been substantial work in the robustness of object
detection algorithms and online metric learning, there is little research on
benchmarking scoring metrics to determine any possible indicators of potential
misclassification. An emphasis is put on exploring the potential of taking
these scoring metrics online in order to allow the AV to make perception-based
decisions given real-time constraints. In this work, we explore which, if any,
metrics act as online indicators of when perception algorithms and object
detectors are failing. Our work provides insight on better design principles
and characteristics of online metrics to accurately evaluate the credibility of
object detectors. Our approach employs non-adversarial and realistic
perturbations to images, on which we evaluate various quantitative metrics. We
found that offline metrics can be designed to account for real-world
corruptions such as poor weather conditions and that the analysis of such
metrics can provide a segue into designing online metrics. This is a clear next
step as it can allow for error-free autonomous vehicle perception and safer
time-critical and safety-critical decision-making.

该研究探讨了自主驾驶汽车中目标检测算法的可靠性问题，提出一种设计在线指标以准确评估目标检测算法的可信度的方法，并采用现实的图像扰动对各种量化指标进行评估。