With the emergence of social media, voluminous video clips are uploaded every
day, and retrieving the most relevant visual content with a language query
becomes critical. Most approaches aim to learn a joint embedding space for
plain textual and visual contents without adequately exploiting their
intra-modality structures and inter-modality correlations. This paper proposes
a novel transformer that explicitly disentangles the text and video into
semantic roles of objects, spatial contexts and temporal contexts with an
attention scheme to learn the intra- and inter-role correlations among the
three roles to discover discriminative features for matching at different
levels. The preliminary results on popular YouCook2 indicate that our approach
surpasses a current state-of-the-art method, with a high margin in all metrics.
It also overpasses two SOTA methods in terms of two metrics.

该论文提出了一种新型的 Transformer 模型，将文本和视频显式分离，通过注意力机制了解三种角色的内部和内部角色之间的相关性，以在不同级别上寻找识别特征，初步结果表明，在所有的指标中，我们的方法都超过了当前最先进的方 法，在两个指标上也超过了两种最先进的方法。