Text-to-video retrieval systems have recently made significant progress by
utilizing pre-trained models trained on large-scale image-text pairs. However,
most of the latest methods primarily focus on the video modality while
disregarding the audio signal for this task. Nevertheless, a recent advancement
by ECLIPSE has improved long-range text-to-video retrieval by developing an
audiovisual video representation. Nonetheless, the objective of the
text-to-video retrieval task is to capture the complementary audio and video
information that is pertinent to the text query rather than simply achieving
better audio and video alignment. To address this issue, we introduce TEFAL, a
TExt-conditioned Feature ALignment method that produces both audio and video
representations conditioned on the text query. Instead of using only an
audiovisual attention block, which could suppress the audio information
relevant to the text query, our approach employs two independent cross-modal
attention blocks that enable the text to attend to the audio and video
representations separately. Our proposed method's efficacy is demonstrated on
four benchmark datasets that include audio: MSR-VTT, LSMDC, VATEX, and
Charades, and achieves better than state-of-the-art performance consistently
across the four datasets. This is attributed to the additional
text-query-conditioned audio representation and the complementary information
it adds to the text-query-conditioned video representation.

通过引入 TEFAL 方法，利用跨模态注意力机制实现了文本查询条件下的音频和视频表示，从而提高了文本到视频检索任务的性能。