In recent years, considerable progress on the task of text-video retrieval
has been achieved by leveraging large-scale pretraining on visual and audio
datasets to construct powerful video encoders. By contrast, despite the natural
symmetry, the design of effective algorithms for exploiting large-scale
language pretraining remains under-explored. In this work, we are the first to
investigate the design of such algorithms and propose a novel generalized
distillation method, TeachText, which leverages complementary cues from
multiple text encoders to provide an enhanced supervisory signal to the
retrieval model. Moreover, we extend our method to video side modalities and
show that we can effectively reduce the number of used modalities at test time
without compromising performance. Our approach advances the state of the art on
several video retrieval benchmarks by a significant margin and adds no
computational overhead at test time. Last but not least, we show an effective
application of our method for eliminating noise from retrieval datasets. Code
and data can be found at this https URL

这篇论文探索了利用大规模预训练的多个文本编码器设计的算法，提出了一种新颖的综合蒸馏方法 TeachText，并将其拓展应用到视频检索上，在多个视频检索基准上超过了现有技术，而且在测试时不会增加计算负荷。