The new generation of state-of-the-art computer vision systems are trained
from natural language supervision, ranging from simple object category names to
descriptive captions. This form of supervision ensures high generality and
usability of the learned visual models, due to the broad concept coverage
achieved via large-scale data collection process. Alternatively, we argue that
learning with external knowledge is a promising way which leverages a much more
structured source of supervision and offers sample efficiency. We propose
K-LITE, a simple strategy to leverage external knowledge for building
transferable visual systems: In training, it enriches entities in text with
WordNet and Wiktionary knowledge, leading to an efficient and scalable approach
to learning image representations that uses knowledge about the visual
concepts. In evaluation, the text is also augmented with external knowledge and
then used to reference learned visual concepts (or describe new ones) to enable
zero-shot and few-shot transfer of the pre-trained models. We study the
performance of K-LITE on two important computer vision problems, image
classification and object detection, benchmarking on 20 and 13 different
existing datasets, respectively. The proposed knowledge-augmented models show
significant improvement in transfer learning performance over existing methods.
Our code is available at this https URL.

该论文探讨了使用外部知识作为计算机视觉系统训练的一种有效方法，并提出了一种名为 K-LITE 的策略，旨在通过使用 WordNet 和 Wiktionary 的知识来丰富文本数据从而提高图像识别和目标检测的性能。在 20 个数据集和 13 个不同的基准测试上，与现有方法相比，使用外部知识的知识增强模型表现出了更高的迁移学习性能。