This report describes our submission to the Ego4D Moment Queries Challenge
2023. Our submission extends ActionFormer, a latest method for temporal action
localization. Our extension combines an improved ground-truth assignment
strategy during training and a refined version of SoftNMS at inference time.
Our solution is ranked 2nd on the public leaderboard with 26.62% average mAP
and 45.69% Recall@1x at tIoU=0.5 on the test set, significantly outperforming
the strong baseline from 2023 challenge. Our code is available at
this https URL

本文介绍了我们团队对 Ego4D Moment Queries Challenge 2023 的提交，扩展了 ActionFormer 方法以实现时间动作定位，并在训练过程中改进了地面真实分配策略和使用了精炼的 SoftNMS 方法，取得了显著的成果。我们的解决方案在测试集上获得平均 mAP 26.62％和 tIoU=0.5 时的 Recall@1x 45.69％，并排在公共排行榜上第二名。

对于 Ego4D Moment 查询，NMS 阈值很重要 -- 2023 Ego4D Moment 查询挑战赛第二名解决方案

NMS Threshold matters for Ego4D Moment Queries -- 2nd place solution to  the Ego4D Moment Queries Challenge 2023

This report describes our submission to the Ego4D Moment Queries Challenge
2022. Our submission builds on ActionFormer, the state-of-the-art backbone for
temporal action localization, and a trio of strong video features from
SlowFast, Omnivore and EgoVLP. Our solution is ranked 2nd on the public
leaderboard with 21.76% average mAP on the test set, which is nearly three
times higher than the official baseline. Further, we obtain 42.54% Recall@1x at
tIoU=0.5 on the test set, outperforming the top-ranked solution by a
significant margin of 1.41 absolute percentage points. Our code is available at
this https URL

我们在 Ego4D Moment Queries Challenge 2022 的比赛中使用了 ActionFormer 作为主体，加上 SlowFast、Omnivore 和 EgoVLP 三种强大的视频特征，我们的解决方案在测试集上的平均 mAP 为 21.76％，是官方基线的三倍，同时我们在 tIoU = 0.5 下获得 42.54％的 Recall @1x， 绝对百分比达 1.41 个百分点的显着优势。

当坚强的骨架遇见强大的特征 —— 用于 Ego4D 时刻查询挑战的 ActionFormer

Where a Strong Backbone Meets Strong Features -- ActionFormer for Ego4D  Moment Queries Challenge

Self-attention based Transformer models have demonstrated impressive results
for image classification and object detection, and more recently for video
understanding. Inspired by this success, we investigate the application of
Transformer networks for temporal action localization in videos. To this end,
we present ActionFormer -- a simple yet powerful model to identify actions in
time and recognize their categories in a single shot, without using action
proposals or relying on pre-defined anchor windows. ActionFormer combines a
multiscale feature representation with local self-attention, and uses a
light-weighted decoder to classify every moment in time and estimate the
corresponding action boundaries. We show that this orchestrated design results
in major improvements upon prior works. Without bells and whistles,
ActionFormer achieves 71.0% mAP at tIoU=0.5 on THUMOS14, outperforming the best
prior model by 14.1 absolute percentage points. Further, ActionFormer
demonstrates strong results on ActivityNet 1.3 (36.6% average mAP) and
EPIC-Kitchens 100 (+13.5% average mAP over prior works). Our code is available
at this http URL

ActionFormer 是一种基于 Transformer 网络的模型，采用了多尺度特征表示和本地自我注意力机制来识别视频中的动作。它在 THUMOS14 上取得了 71.0％ mAP，在 ActivityNet 1.3 和 EPIC-Kitchens 100 中也表现出色。