In this paper, we address web-scale visual entity recognition, specifically
the task of mapping a given query image to one of the 6 million existing
entities in Wikipedia. One way of approaching a problem of such scale is using
dual-encoder models (eg CLIP), where all the entity names and query images are
embedded into a unified space, paving the way for an approximate k-NN search.
Alternatively, it is also possible to re-purpose a captioning model to directly
generate the entity names for a given image. In contrast, we introduce a novel
Generative Entity Recognition (GER) framework, which given an input image
learns to auto-regressively decode a semantic and discriminative ``code''
identifying the target entity. Our experiments demonstrate the efficacy of this
GER paradigm, showcasing state-of-the-art performance on the challenging OVEN
benchmark. GER surpasses strong captioning, dual-encoder, visual matching and
hierarchical classification baselines, affirming its advantage in tackling the
complexities of web-scale recognition.

本文旨在探讨网络规模的视觉实体识别，介绍了一个名为 GER 的新型实体识别框架，通过学习自动解码一个语义和区别性的 “编码” 来识别目标实体，并证明其在应对网络规模识别复杂性方面具有优势。

一种面向维基百科规模的视觉实体识别的生成方法

A Generative Approach for Wikipedia-Scale Visual Entity Recognition

Dual-encoder models have demonstrated significant success in dense retrieval
tasks for open-domain question answering that mostly involves zero-shot and
few-shot scenarios. However, their performance in many-shot retrieval problems
where training data is abundant, such as extreme multi-label classification
(XMC), remains under-explored. Existing empirical evidence suggests that, for
such problems, the dual-encoder method's accuracies lag behind the performance
of state-of-the-art (SOTA) extreme classification methods that grow the number
of learnable parameters linearly with the number of classes. As a result, some
recent extreme classification techniques use a combination of dual-encoders and
a learnable classification head for each class to excel on these tasks. In this
paper, we investigate the potential of "pure" DE models in XMC tasks. Our
findings reveal that when trained correctly standard dual-encoders can match or
outperform SOTA extreme classification methods by up to 2% at Precision@1 even
on the largest XMC datasets while being 20x smaller in terms of the number of
trainable parameters. We further propose a differentiable topk error-based loss
function, which can be used to specifically optimize for Recall@k metrics. We
include our PyTorch implementation along with other resources for reproducing
the results in the supplementary material.

通过研究表明，经过正确训练的标准双编码器模型在极限多标签分类问题中可以匹配或超越状态 - of-the-art 极限分类方法在 Precision@1 方面的性能，并且在可训练参数的数量上小 20 倍。

双编码器在极端多标签分类中的效果

Efficacy of Dual-Encoders for Extreme Multi-Label Classification

In this paper, we propose a new dense retrieval model which learns diverse
document representations with deep query interactions. Our model encodes each
document with a set of generated pseudo-queries to get query-informed,
multi-view document representations. It not only enjoys high inference
efficiency like the vanilla dual-encoder models, but also enables deep
query-document interactions in document encoding and provides multi-faceted
representations to better match different queries. Experiments on several
benchmarks demonstrate the effectiveness of the proposed method, out-performing
strong dual encoder baselines.The code is available at
\url{this https URL

提出了一种新的密集检索模型，使用深度查询交互学习多样的文档表示，通过生成的伪查询对每个文档进行编码，以获得查询信息和多视角的文档表示，不仅具有类似于原始双编码器模型的高推理效率，还可以在文档编码中实现深入的查询 - 文档交互并提供多方面的表示来更好地匹配不同的查询。