Large-scale retrieval is to recall relevant documents from a huge collection
given a query. It relies on representation learning to embed documents and
queries into a common semantic encoding space. According to the encoding space,
recent retrieval methods based on pre-trained language models (PLM) can be
coarsely categorized into either dense-vector or lexicon-based paradigms. These
two paradigms unveil the PLMs' representation capability in different
granularities, i.e., global sequence-level compression and local word-level
contexts, respectively. Inspired by their complementary global-local
contextualization and distinct representing views, we propose a new learning
framework, UnifieR, which unifies dense-vector and lexicon-based retrieval in
one model with a dual-representing capability. Experiments on passage retrieval
benchmarks verify its effectiveness in both paradigms. A uni-retrieval scheme
is further presented with even better retrieval quality. We lastly evaluate the
model on BEIR benchmark to verify its transferability.

本论文提出了一种名为 UnifieR 的学习框架，它将基于预训练语言模型的密集向量和基于词汇表的检索统一在一个模型中，并在多个基准测试中验证了其有效性和可转移性。

UnifieR: 大规模信息检索统一检索器

UnifieR: A Unified Retriever for Large-Scale Retrieval

The task of large-scale retrieval-based image localization is to estimate the
geographical location of a query image by recognizing its nearest reference
images from a city-scale dataset. However, the general public benchmarks only
provide noisy GPS labels associated with the training images, which act as weak
supervisions for learning image-to-image similarities. Such label noise
prevents deep neural networks from learning discriminative features for
accurate localization. To tackle this challenge, we propose to self-supervise
image-to-region similarities in order to fully explore the potential of
difficult positive images alongside their sub-regions. The estimated
image-to-region similarities can serve as extra training supervision for
improving the network in generations, which could in turn gradually refine the
fine-grained similarities to achieve optimal performance. Our proposed
self-enhanced image-to-region similarity labels effectively deal with the
training bottleneck in the state-of-the-art pipelines without any additional
parameters or manual annotations in both training and inference. Our method
outperforms state-of-the-arts on the standard localization benchmarks by
noticeable margins and shows excellent generalization capability on multiple
image retrieval datasets.

通过自监督学习图像的区域相似性，实现了对具有潜在难度的正面图像及其子区域的充分探索，并且在训练和推理中没有额外的参数或手动注释，能够有效地处理当前现有管道中的训练瓶颈

大规模图像定位的自监督细粒度区域相似性

Self-supervising Fine-grained Region Similarities for Large-scale Image  Localization

In this paper, we investigate the problem of zero-shot sketch-based image
retrieval (ZS-SBIR), where human sketches are used as queries to conduct
retrieval of photos from unseen categories. We importantly advance prior arts
by proposing a novel ZS-SBIR scenario that represents a firm step forward in
its practical application. The new setting uniquely recognizes two important
yet often neglected challenges of practical ZS-SBIR, (i) the large domain gap
between amateur sketch and photo, and (ii) the necessity for moving towards
large-scale retrieval. We first contribute to the community a novel ZS-SBIR
dataset, QuickDraw-Extended, that consists of 330,000 sketches and 204,000
photos spanning across 110 categories. Highly abstract amateur human sketches
are purposefully sourced to maximize the domain gap, instead of ones included
in existing datasets that can often be semi-photorealistic. We then formulate a
ZS-SBIR framework to jointly model sketches and photos into a common embedding
space. A novel strategy to mine the mutual information among domains is
specifically engineered to alleviate the domain gap. External semantic
knowledge is further embedded to aid semantic transfer. We show that, rather
surprisingly, retrieval performance significantly outperforms that of
state-of-the-art on existing datasets that can already be achieved using a
reduced version of our model. We further demonstrate the superior performance
of our full model by comparing with a number of alternatives on the newly
proposed dataset. The new dataset, plus all training and testing code of our
model, will be publicly released to facilitate future research

本文提出了一种新的零样本基于草图的图像检索 (ZS-SBIR) 场景，该场景能够有效解决 ZS-SBIR 实际应用中常见的大领域差异和大规模检索问题，提供了一个全新的 ZS-SBIR 数据集 QuickDraw-Extended，采用了一种新策略来处理领域差异，同时集成了外部语义知识以帮助语义转换，实验表明该模型在现有数据集上表现优异，并提供训练代码和数据集以供未来研究使用。

Doodle to Search: 实用的零样本基于草图的图像检索

Doodle to Search: Practical Zero-Shot Sketch-based Image Retrieval

Recent advances in the machine learning community allowed different use cases
to emerge, as its association to domains like cooking which created the
computational cuisine. In this paper, we tackle the picture-recipe alignment
problem, having as target application the large-scale retrieval task (finding a
recipe given a picture, and vice versa). Our approach is validated on the
Recipe1M dataset, composed of one million image-recipe pairs and additional
class information, for which we achieve state-of-the-art results.

本文基于机器学习技术，提出了一种用于大规模检索任务的图片 - 食谱对齐方法，并在 Recipe1M 数据集上进行了实验，取得了最先进的成果。