This technical report presents our solution for temporal action detection
task in AcitivityNet Challenge 2021. The purpose of this task is to locate and
identify actions of interest in long untrimmed videos. The crucial challenge of
the task comes from that the temporal duration of action varies dramatically,
and the target actions are typically embedded in a background of irrelevant
activities. Our solution builds on BMN, and mainly contains three steps: 1)
action classification and feature encoding by Slowfast, CSN and ViViT; 2)
proposal generation. We improve BMN by embedding the proposed Proposal Relation
Network (PRN), by which we can generate proposals of high quality; 3) action
detection. We calculate the detection results by assigning the proposals with
corresponding classification results. Finally, we ensemble the results under
different settings and achieve 44.7% on the test set, which improves the
champion result in ActivityNet 2020 by 1.9% in terms of average mAP.

本技术报告介绍了我们在 AcitivityNet 2021 挑战赛中针对时间动作检测任务提出的解决方案。我们通过使用 BMN、slowfast、CSN 和 ViViT 等方法来进行动作分类和特征编码，改进 BMN 算法的提案生成和行为检测以及不同设置下的结果综合，最终在测试集中取得 44.7% 的平均 mAP，相较于 ActivityNet 2020 冠军成绩提高了 1.9%。

时间动作检测的关系网络提案

Proposal Relation Network for Temporal Action Detection

This technical report analyzes a temporal action localization method we used
in the HACS competition which is hosted in Activitynet Challenge 2020.The goal
of our task is to locate the start time and end time of the action in the
untrimmed video, and predict action category.Firstly, we utilize the
video-level feature information to train multiple video-level action
classification models. In this way, we can get the category of action in the
video.Secondly, we focus on generating high quality temporal proposals.For this
purpose, we apply BMN to generate a large number of proposals to obtain high
recall rates. We then refine these proposals by employing a cascade structure
network called Refine Network, which can predict position offset and new IOU
under the supervision of ground truth.To make the proposals more accurate, we
use bidirectional LSTM, Nonlocal and Transformer to capture temporal
relationships between local features of each proposal and global features of
the video data.Finally, by fusing the results of multiple models, our method
obtains 40.55% on the validation set and 40.53% on the test set in terms of
mAP, and achieves Rank 1 in this challenge.

本技术报告分析了我们在 Activitynet Challenge 2020 中使用的一种时间动作定位方法，该方法利用视频级别特征信息训练多个视频级别动作分类模型，并应用 BMN 等方法产生高质量的时间提案，再通过级联结构网络 Refine Network 和多种模型融合方法，实现了在 HACS 比赛中 Rank1 的好成绩。

时间融合网络用于时序动作定位：提交至 ActivityNet Challenge 2020（E 任务）

Temporal Fusion Network for Temporal Action Localization:Submission to  ActivityNet Challenge 2020 (Task E)

Temporal action proposal generation is an challenging and promising task
which aims to locate temporal regions in real-world videos where action or
event may occur. Current bottom-up proposal generation methods can generate
proposals with precise boundary, but cannot efficiently generate adequately
reliable confidence scores for retrieving proposals. To address these
difficulties, we introduce the Boundary-Matching (BM) mechanism to evaluate
confidence scores of densely distributed proposals, which denote a proposal as
a matching pair of starting and ending boundaries and combine all densely
distributed BM pairs into the BM confidence map. Based on BM mechanism, we
propose an effective, efficient and end-to-end proposal generation method,
named Boundary-Matching Network (BMN), which generates proposals with precise
temporal boundaries as well as reliable confidence scores simultaneously. The
two-branches of BMN are jointly trained in an unified framework. We conduct
experiments on two challenging datasets: THUMOS-14 and ActivityNet-1.3, where
BMN shows significant performance improvement with remarkable efficiency and
generalizability. Further, combining with existing action classifier, BMN can
achieve state-of-the-art temporal action detection performance.

提出一种名为 “Boundary-Matching Network（BMN）” 的有效、高效、端对端的建议生成方法，可以同时生成具有精确时间边界和可靠置信度分数的建议，提出 BM 机制来评估密集分布的建议的置信度得分，以及将所有密集分布的 BM 对结合成 BM 置信度图，测试结果在 THUMOS-14 和 ActivityNet-1.3 数据集上表现出一定优越性。