An important goal of computer vision is to build systems that learn visual
representations over time that can be applied to many tasks. In this paper, we
investigate a vision-language embedding as a core representation and show that
it leads to better cross-task transfer than standard multi-task learning. In
particular, the task of visual recognition is aligned to the task of visual
question answering by forcing each to use the same word-region embeddings. We
show this leads to greater inductive transfer from recognition to VQA than
standard multitask learning. Visual recognition also improves, especially for
categories that have relatively few recognition training labels but appear
often in the VQA setting. Thus, our paper takes a small step towards creating
more general vision systems by showing the benefit of interpretable, flexible,
and trainable core representations.

本文研究视觉特征的表征方式，并探讨了通过将视觉识别任务和视觉问答任务各自对应的语言表示方式建立联系，以实现更好的跨任务数据迁移的方法。结果表明，这种方法比常规的多任务学习方法更有效，并且能够大幅提高在视觉识别任务和视觉问答任务上的准确率。