Deep metric learning (DML) enables learning with less supervision through its
emphasis on the similarity structure of representations. There has been much
work on improving generalization of DML in settings like zero-shot retrieval,
but little is known about its implications for fairness. In this paper, we are
the first to evaluate state-of-the-art DML methods trained on imbalanced data,
and to show the negative impact these representations have on minority subgroup
performance when used for downstream tasks. In this work, we first define
fairness in DML through an analysis of three properties of the representation
space -- inter-class alignment, intra-class alignment, and uniformity -- and
propose finDML, the fairness in non-balanced DML benchmark to characterize
representation fairness. Utilizing finDML, we find bias in DML representations
to propagate to common downstream classification tasks. Surprisingly, this bias
is propagated even when training data in the downstream task is re-balanced. To
address this problem, we present Partial Attribute De-correlation (PARADE) to
de-correlate feature representations from sensitive attributes and reduce
performance gaps between subgroups in both embedding space and downstream
metrics.

本文首次针对非平衡数据进行深度度量学习（DML）方法的公平性评估，提出了 finDML 工具来表征 DML 的公平性并发现了 DML 的子集表现的表示偏差，而 Partial Attribute De-correlation (PARADE) 则减少了对敏感属性的相关性并减小了表征空间中的性能差距。

深度学习中公平性只是度量吗？评估和解决子组差距

Is Fairness Only Metric Deep? Evaluating and Addressing Subgroup Gaps in Deep Metric Learning

We present a new benchmark dataset called PARADE for paraphrase
identification that requires specialized domain knowledge. PARADE contains
paraphrases that overlap very little at the lexical and syntactic level but are
semantically equivalent based on computer science domain knowledge, as well as
non-paraphrases that overlap greatly at the lexical and syntactic level but are
not semantically equivalent based on this domain knowledge. Experiments show
that both state-of-the-art neural models and non-expert human annotators have
poor performance on PARADE. For example, BERT after fine-tuning achieves an F1
score of 0.709, which is much lower than its performance on other paraphrase
identification datasets. PARADE can serve as a resource for researchers
interested in testing models that incorporate domain knowledge. We make our
data and code freely available.

本文提出了一个名为 PARADE 的新型基准数据集，用于测试包含不同领域专业知识的文本相似性鉴别模型。结果表明，即使是最先进的神经网络模型以及普通人类标注者在此数据集上的表现也十分糟糕。PARADE 可以为需要加入专业领域知识的模型提供一个资源。我们公开了数据集和代码。

PARADE: 面向计算机科学领域知识的近义识别新数据集

PARADE: A New Dataset for Paraphrase Identification Requiring Computer  Science Domain Knowledge

Pretrained transformer models, such as BERT and T5, have shown to be highly
effective at ad-hoc passage and document ranking. Due to inherent sequence
length limits of these models, they need to be run over a document's passages,
rather than processing the entire document sequence at once. Although several
approaches for aggregating passage-level signals have been proposed, there has
yet to be an extensive comparison of these techniques. In this work, we explore
strategies for aggregating relevance signals from a document's passages into a
final ranking score. We find that passage representation aggregation techniques
can significantly improve over techniques proposed in prior work, such as
taking the maximum passage score. We call this new approach PARADE. In
particular, PARADE can significantly improve results on collections with broad
information needs where relevance signals can be spread throughout the document
(such as TREC Robust04 and GOV2). Meanwhile, less complex aggregation
techniques may work better on collections with an information need that can
often be pinpointed to a single passage (such as TREC DL and TREC Genomics). We
also conduct efficiency analyses, and highlight several strategies for
improving transformer-based aggregation.

本文研究了如何将文档的段落级别的相关信号聚合成最终的排名分数，提出了新的 PARADE 方法，证明了它可以显著提高广泛信息需求的文集的结果，同时，也探索了提高基于 Transformer 聚合效率的方法。