Vision language tasks, such as answering questions about or generating
captions that describe an image, are difficult tasks for computers to perform.
A relatively recent body of research has adapted the pretrained transformer
architecture introduced in \citet{vaswani2017attention} to vision language
modeling. Transformer models have greatly improved performance and versatility
over previous vision language models. They do so by pretraining models on a
large generic datasets and transferring their learning to new tasks with minor
changes in architecture and parameter values. This type of transfer learning
has become the standard modeling practice in both natural language processing
and computer vision. Vision language transformers offer the promise of
producing similar advancements in tasks which require both vision and language.
In this paper, we provide a broad synthesis of the currently available research
on vision language transformer models and offer some analysis of their
strengths, limitations and some open questions that remain.

视觉语言任务中，基于预训练的变压器架构在视觉语言建模方面表现出色，为视觉和语言结合的任务带来了类似的进展。