Open-ended video question answering aims to automatically generate the
natural-language answer from referenced video contents according to the given
question. Currently, most existing approaches focus on short-form video
question answering with multi-modal recurrent encoder-decoder networks.
Although these works have achieved promising performance, they may still be
ineffectively applied to long-form video question answering due to the lack of
long-range dependency modeling and the suffering from the heavy computational
cost. To tackle these problems, we propose a fast Hierarchical Convolutional
Self-Attention encoder-decoder network(HCSA). Concretely, we first develop a
hierarchical convolutional self-attention encoder to efficiently model
long-form video contents, which builds the hierarchical structure for video
sequences and captures question-aware long-range dependencies from video
context. We then devise a multi-scale attentive decoder to incorporate
multi-layer video representations for answer generation, which avoids the
information missing of the top encoder layer. The extensive experiments show
the effectiveness and efficiency of our method.

本论文提出了一种高效的分层卷积自注意编 - 解码神经网络，旨在解决长形视频问答中长程依赖建模和计算成本高的问题。该方法通过分层结构和多通道视频表示的注意力机制，在视频序列中捕捉基于问题的长程依赖关系，提高了回答的效率和准确性。