We propose Encyclopedic-VQA, a large scale visual question answering (VQA)
dataset featuring visual questions about detailed properties of fine-grained
categories and instances. It contains 221k unique question+answer pairs each
matched with (up to) 5 images, resulting in a total of 1M VQA samples.
Moreover, our dataset comes with a controlled knowledge base derived from
Wikipedia, marking the evidence to support each answer. Empirically, we show
that our dataset poses a hard challenge for large vision+language models as
they perform poorly on our dataset: PaLI [14] is state-of-the-art on OK-VQA
[37], yet it only achieves 13.0% accuracy on our dataset. Moreover, we
experimentally show that progress on answering our encyclopedic questions can
be achieved by augmenting large models with a mechanism that retrieves relevant
information from the knowledge base. An oracle experiment with perfect
retrieval achieves 87.0% accuracy on the single-hop portion of our dataset, and
an automatic retrieval-augmented prototype yields 48.8%. We believe that our
dataset enables future research on retrieval-augmented vision+language models.

提出了一个大规模的视觉问答数据集 Encyclopedic-VQA，包括涉及细粒度类别和实例详细属性的视觉问题，并配有来自 Wikipedia 的知识库作为支持每个答案的证据，可以通过检索增强模型实现对这些问答的准确回答。

百科问答视觉：有关细粒度类别详细属性的视觉问题

Encyclopedic VQA: Visual questions about detailed properties of  fine-grained categories

Generating keyphrases that summarize the main points of a document is a
fundamental task in natural language processing. Although existing generative
models are capable of predicting multiple keyphrases for an input document as
well as determining the number of keyphrases to generate, they still suffer
from the problem of generating too few keyphrases. To address this problem, we
propose a reinforcement learning (RL) approach for keyphrase generation, with
an adaptive reward function that encourages a model to generate both sufficient
and accurate keyphrases. Furthermore, we introduce a new evaluation method that
incorporates name variations of the ground-truth keyphrases using the Wikipedia
knowledge base. Thus, our evaluation method can more robustly evaluate the
quality of predicted keyphrases. Extensive experiments on five real-world
datasets of different scales demonstrate that our RL approach consistently and
significantly improves the performance of the state-of-the-art generative
models with both conventional and new evaluation methods.

本文提出了一种使用强化学习方法进行关键词生成的技术，利用自适应的奖励函数鼓励模型生成足够且准确的关键词，并引入了一种新的评估方法，该方法使用维基百科知识库来确保更加鲁棒的关键词质量评估。该方法在不同规模的五个真实世界数据集上实现了显著的性能提升，从而验证了其优越性。