Pre-trained contextualized embedding models such as BERT are a standard
building block in many natural language processing systems. We demonstrate that
the sentence-level representations produced by some off-the-shelf
contextualized embedding models have a narrow distribution in the embedding
space, and thus perform poorly for the task of identifying semantically similar
questions in real-world English business conversations. We describe a method
that uses appropriately tuned representations and a small set of exemplars to
group questions of interest to business users in a visualization that can be
used for data exploration or employee coaching.

我们使用经过适当调整的表示方法和少量示例来对商业用户感兴趣的问题进行分组，并制作可用于数据探索或员工培训的可视化。

从自然出现的商业对话中提取相似问题

Extracting Similar Questions From Naturally-occurring Business Conversations

Concerns regarding the footprint of societal biases in information retrieval
(IR) systems have been raised in several previous studies. In this work, we
examine various recent IR models from the perspective of the degree of gender
bias in their retrieval results. To this end, we first provide a bias
measurement framework which includes two metrics to quantify the degree of the
unbalanced presence of gender-related concepts in a given IR model's ranking
list. To examine IR models by means of the framework, we create a dataset of
non-gendered queries, selected by human annotators. Applying these queries to
the MS MARCO Passage retrieval collection, we then measure the gender bias of a
BM25 model and several recent neural ranking models. The results show that
while all models are strongly biased toward male, the neural models, and in
particular the ones based on contextualized embedding models, significantly
intensify gender bias. Our experiments also show an overall increase in the
gender bias of neural models when they exploit transfer learning, namely when
they use (already biased) pre-trained embeddings.

本研究以非性别查询为基础，根据两个度量标准，从性别偏见的角度研究了 BM25 模型和几种最近的神经排名模型，实验表明，虽然所有模型都强烈偏向男性，但神经模型，特别是基于语境嵌入模型的神经模型，显著加剧了性别偏见。而且当这些神经模型利用传递学习时，性别偏见总体上呈增加趋势。