Detecting meaningful events in an untrimmed video is essential for dense
video captioning. In this work, we propose a novel and simple model for event
sequence generation and explore temporal relationships of the event sequence in
the video. The proposed model omits inefficient two-stage proposal generation
and directly generates event boundaries conditioned on bi-directional temporal
dependency in one pass. Experimental results show that the proposed event
sequence generation model can generate more accurate and diverse events within
a small number of proposals. For the event captioning, we follow our previous
work to employ the intra-event captioning models into our pipeline system. The
overall system achieves state-of-the-art performance on the dense-captioning
events in video task with 9.894 METEOR score on the challenge testing set.

提出了一种用于密集视频字幕的事件序列生成模型，通过双向时间依赖在一次传递中生成事件边界来探索视频中的时间关系，避免了低效的两阶段提议生成，并在事件字幕模型中取得了最新的性能。

清华大学人工智能联盟团队 2020 Activitynet 任务二技术报告：探索密集视频字幕的顺序事件检测

Team RUC_AIM3 Technical Report at Activitynet 2020 Task 2: Exploring  Sequential Events Detection for Dense Video Captioning

Dense video captioning is an extremely challenging task since accurate and
coherent description of events in a video requires holistic understanding of
video contents as well as contextual reasoning of individual events. Most
existing approaches handle this problem by first detecting event proposals from
a video and then captioning on a subset of the proposals. As a result, the
generated sentences are prone to be redundant or inconsistent since they fail
to consider temporal dependency between events. To tackle this challenge, we
propose a novel dense video captioning framework, which models temporal
dependency across events in a video explicitly and leverages visual and
linguistic context from prior events for coherent storytelling. This objective
is achieved by 1) integrating an event sequence generation network to select a
sequence of event proposals adaptively, and 2) feeding the sequence of event
proposals to our sequential video captioning network, which is trained by
reinforcement learning with two-level rewards at both event and episode levels
for better context modeling. The proposed technique achieves outstanding
performances on ActivityNet Captions dataset in most metrics.

本文提出了一种新颖的密集视频字幕框架，它通过显式建模视频中事件的时间依赖性并利用先前事件的视觉和语言上下文来实现连贯的叙述。该框架由事件序列生成网络和序列视频字幕网络组成，后者利用强化学习进行训练，并在事件和剧集两个级别上进行两级奖励以实现更好的上下文建模。在大多数指标上，该方法在 ActivityNet Captions 数据集上取得了出色的表现。