The objective of this work is the effective extraction of spatial and dynamic
features for Continuous Sign Language Recognition (CSLR). To accomplish this,
we utilise a two-pathway SlowFast network, where each pathway operates at
distinct temporal resolutions to separately capture spatial (hand shapes,
facial expressions) and dynamic (movements) information. In addition, we
introduce two distinct feature fusion methods, carefully designed for the
characteristics of CSLR: (1) Bi-directional Feature Fusion (BFF), which
facilitates the transfer of dynamic semantics into spatial semantics and vice
versa; and (2) Pathway Feature Enhancement (PFE), which enriches dynamic and
spatial representations through auxiliary subnetworks, while avoiding the need
for extra inference time. As a result, our model further strengthens spatial
and dynamic representations in parallel. We demonstrate that the proposed
framework outperforms the current state-of-the-art performance on popular CSLR
datasets, including PHOENIX14, PHOENIX14-T, and CSL-Daily.

本研究旨在通过双通道 SlowFast 网络以及特殊的特征融合方法，有效提取连续手语识别中的时空特征，并在多个数据集上展示了该模型在空间和动态表示方面的优势。

SlowFast 网络用于连续手语识别

SlowFast Network for Continuous Sign Language Recognition

In this report, we introduce our adaptation of image-text models for
long-term action anticipation. Our Video + CLIP framework makes use of a
large-scale pre-trained paired image-text model: CLIP and a video encoder
Slowfast network. The CLIP embedding provides fine-grained understanding of
objects relevant for an action whereas the slowfast network is responsible for
modeling temporal information within a video clip of few frames. We show that
the features obtained from both encoders are complementary to each other, thus
outperforming the baseline on Ego4D for the task of long-term action
anticipation. Our code is available at
github.com/srijandas07/clip_baseline_LTA_Ego4d.

本文报道了我们在长期行动预测方面使用图像 - 文本模型的调整。我们的视频 + CLIP 框架利用了大规模预先训练的成对图像 - 文本模型：CLIP 和视频编码器 Slowfast 网络。两个编码器得到的特征互补，因此在 Ego4D 上的长期行动预测任务中优于基线。我们的代码位于 github.com/srijandas07/clip_baseline_LTA_Ego4d。

Ego4D 长期动作预测的 Video + CLIP 基线

Video + CLIP Baseline for Ego4D Long-term Action Anticipation

This paper is a brief report to our submission to the VIPriors Action
Recognition Challenge. Action recognition has attracted many researchers
attention for its full application, but it is still challenging. In this paper,
we study previous methods and propose our method. In our method, we are
primarily making improvements on the SlowFast Network and fusing with TSM to
make further breakthroughs. Also, we use a fast but effective way to extract
motion features from videos by using residual frames as input. Better motion
features can be extracted using residual frames with SlowFast, and the
residual-frame-input path is an excellent supplement for existing
RGB-frame-input models. And better performance obtained by combining 3D
convolution(SlowFast) with 2D convolution(TSM). The above experiments were all
trained from scratch on UCF101.

本文研究行动识别问题，结合 SlowFast 网络和 TSM 方法，在视频中提取运动特征可以使用残差帧作为输入和 RGB 帧输入模型的优秀补充，通过 3D 卷。积（SlowFast）与 2D 卷积（TSM）的组合取得更好的性能结果，以上实验均在 UCF101 中从头开始训练。