Aligning a user query and video clips in cross-modal latent space and that
with semantic concepts are two mainstream approaches for ad-hoc video search
(AVS). However, the effectiveness of existing approaches is bottlenecked by the
small sizes of available video-text datasets and the low quality of concept
banks, which results in the failures of unseen queries and the
out-of-vocabulary problem. This paper addresses these two problems by
constructing a new dataset and developing a multi-word concept bank.
Specifically, capitalizing on a generative model, we construct a new dataset
consisting of 7 million generated text and video pairs for pre-training. To
tackle the out-of-vocabulary problem, we develop a multi-word concept bank
based on syntax analysis to enhance the capability of a state-of-the-art
interpretable AVS method in modeling relationships between query words. We also
study the impact of current advanced features on the method. Experimental
results show that the integration of the above-proposed elements doubles the
R@1 performance of the AVS method on the MSRVTT dataset and improves the xinfAP
on the TRECVid AVS query sets for 2016-2023 (eight years) by a margin from 2%
to 77%, with an average about 20%.

通过构建新数据集和发展多词概念库，本文解决了现有方法在出现未见查询和词汇量问题上的瓶颈，实验结果显示以上所述元素的整合将 AVS 方法在 MSRVTT 数据集上的 R@1 性能翻倍，并将在 2016-2023 年（八年）TRECVid AVS 查询集的 xinfAP 增加了 2% 到 77%，平均约为 20%。

通过生成式标题和多词概念库改进用于即席视频搜索的可解释嵌入

Improving Interpretable Embeddings for Ad-hoc Video Search with  Generative Captions and Multi-word Concept Bank

For quantifying progress in Ad-hoc Video Search (AVS), the annual TRECVID AVS
task is an important international evaluation. Solutions submitted by the task
participants vary in terms of their choices of cross-modal matching models,
visual features and training data. As such, what one may conclude from the
evaluation is at a high level that is insufficient to reveal the influence of
the individual components. In order to bridge the gap between the current
solution-level comparison and the desired component-wise comparison, we propose
in this paper a large-scale and systematic evaluation on TRECVID. By selected
combinations of state-of-the-art matching models, visual features and
(pre-)training data, we construct a set of 25 different solutions and evaluate
them on the TRECVID AVS tasks 2016--2020. The presented evaluation helps answer
the key question of what matters for AVS. The resultant observations and
learned lessons are also instructive for developing novel AVS solutions.

通过 TRECVID AVS 任务的大规模系统评估，探讨了各种匹配模型、视觉特征和训练数据与 Ad-hoc Video Search 的关系以及对未来新技术的指导意义。

关于 Ad-hoc 视频搜索的重要因素？TRECVID 上的大规模评估

What Matters for Ad-hoc Video Search? A Large-scale Evaluation on  TRECVID

Retrieving unlabeled videos by textual queries, known as Ad-hoc Video Search
(AVS), is a core theme in multimedia data management and retrieval. The success
of AVS counts on cross-modal representation learning that encodes both query
sentences and videos into common spaces for semantic similarity computation.
Inspired by the initial success of previously few works in combining multiple
sentence encoders, this paper takes a step forward by developing a new and
general method for effectively exploiting diverse sentence encoders. The
novelty of the proposed method, which we term Sentence Encoder Assembly (SEA),
is two-fold. First, different from prior art that use only a single common
space, SEA supports text-video matching in multiple encoder-specific common
spaces. Such a property prevents the matching from being dominated by a
specific encoder that produces an encoding vector much longer than other
encoders. Second, in order to explore complementarities among the individual
common spaces, we propose multi-space multi-loss learning. As extensive
experiments on four benchmarks (MSR-VTT, TRECVID AVS 2016-2019, TGIF and MSVD)
show, SEA surpasses the state-of-the-art. In addition, SEA is extremely ease to
implement. All this makes SEA an appealing solution for AVS and promising for
continuously advancing the task by harvesting new sentence encoders.

本研究提出了一种名为 Sentence Encoder Assembly 的新方法，通过多空间多损失学习实现语句编码器的有效利用和文本 - 视频匹配，并在四个基准测试中表现出优于当前最先进技术的性能。