This paper focuses on open-ended video question answering, which aims to find
the correct answers from a large answer set in response to a video-related
question. This is essentially a multi-label classification task, since a
question may have multiple answers. However, due to annotation costs, the
labels in existing benchmarks are always extremely insufficient, typically one
answer per question. As a result, existing works tend to directly treat all the
unlabeled answers as negative labels, leading to limited ability for
generalization. In this work, we introduce a simple yet effective ranking
distillation framework (RADI) to mitigate this problem without additional
manual annotation. RADI employs a teacher model trained with incomplete labels
to generate rankings for potential answers, which contain rich knowledge about
label priority as well as label-associated visual cues, thereby enriching the
insufficient labeling information. To avoid overconfidence in the imperfect
teacher model, we further present two robust and parameter-free ranking
distillation approaches: a pairwise approach which introduces adaptive soft
margins to dynamically refine the optimization constraints on various pairwise
rankings, and a listwise approach which adopts sampling-based partial listwise
learning to resist the bias in teacher ranking. Extensive experiments on five
popular benchmarks consistently show that both our pairwise and listwise RADIs
outperform state-of-the-art methods. Further analysis demonstrates the
effectiveness of our methods on the insufficient labeling problem.

本文针对开放式视频问题回答进行了研究，提出了一种简单而有效的排名蒸馏框架 (RADI)，用于解决现有标注问题的缺陷，并且在五个流行基准测试中与最先进的方法相比，表现出色。

不足标签的开放式视频问题回答的排名蒸馏

Ranking Distillation for Open-Ended Video Question Answering with  Insufficient Labels

Open-ended video question answering aims to automatically generate the
natural-language answer from referenced video contents according to the given
question. Currently, most existing approaches focus on short-form video
question answering with multi-modal recurrent encoder-decoder networks.
Although these works have achieved promising performance, they may still be
ineffectively applied to long-form video question answering due to the lack of
long-range dependency modeling and the suffering from the heavy computational
cost. To tackle these problems, we propose a fast Hierarchical Convolutional
Self-Attention encoder-decoder network(HCSA). Concretely, we first develop a
hierarchical convolutional self-attention encoder to efficiently model
long-form video contents, which builds the hierarchical structure for video
sequences and captures question-aware long-range dependencies from video
context. We then devise a multi-scale attentive decoder to incorporate
multi-layer video representations for answer generation, which avoids the
information missing of the top encoder layer. The extensive experiments show
the effectiveness and efficiency of our method.

本论文提出了一种高效的分层卷积自注意编 - 解码神经网络，旨在解决长形视频问答中长程依赖建模和计算成本高的问题。该方法通过分层结构和多通道视频表示的注意力机制，在视频序列中捕捉基于问题的长程依赖关系，提高了回答的效率和准确性。