Large-scale language-image pre-trained models (e.g., CLIP) have shown
superior performances on many cross-modal retrieval tasks. However, the problem
of transferring the knowledge learned from such models to video-based person
re-identification (ReID) has barely been explored. In addition, there is a lack
of decent text descriptions in current ReID benchmarks. To address these
issues, in this work, we propose a novel one-stage text-free CLIP-based
learning framework named TF-CLIP for video-based person ReID. More
specifically, we extract the identity-specific sequence feature as the
CLIP-Memory to replace the text feature. Meanwhile, we design a
Sequence-Specific Prompt (SSP) module to update the CLIP-Memory online. To
capture temporal information, we further propose a Temporal Memory Diffusion
(TMD) module, which consists of two key components: Temporal Memory
Construction (TMC) and Memory Diffusion (MD). Technically, TMC allows the
frame-level memories in a sequence to communicate with each other, and to
extract temporal information based on the relations within the sequence. MD
further diffuses the temporal memories to each token in the original features
to obtain more robust sequence features. Extensive experiments demonstrate that
our proposed method shows much better results than other state-of-the-art
methods on MARS, LS-VID and iLIDS-VID. The code is available at
this https URL

提出了一种基于 CLIP 的无文本学习框架，用于基于视频的人员重新识别，通过提取身份特定的序列特征作为 CLIP-Memory，并使用 SSP 模块在线更新 CLIP-Memory，进一步通过 TMD 模块捕获时间信息，取得了比其他最先进方法更好的结果。

TF-CLIP: 学习无须文本的 CLIP 用于基于视频的人物重新识别

TF-CLIP: Learning Text-free CLIP for Video-based Person  Re-Identification

Video-based person reID is an important task, which has received much
attention in recent years due to the increasing demand in surveillance and
camera networks. A typical video-based person reID system consists of three
parts: an image-level feature extractor (e.g. CNN), a temporal modeling method
to aggregate temporal features and a loss function. Although many methods on
temporal modeling have been proposed, it is hard to directly compare these
methods, because the choice of feature extractor and loss function also have a
large impact on the final performance. We comprehensively study and compare
four different temporal modeling methods (temporal pooling, temporal attention,
RNN and 3D convnets) for video-based person reID. We also propose a new
attention generation network which adopts temporal convolution to extract
temporal information among frames. The evaluation is done on the MARS dataset,
and our methods outperform state-of-the-art methods by a large margin. Our
source codes are released at this https URL

该文综合研究和比较了视频人员重识别的四种不同的时间建模方法，包括时间汇聚，时间关注，RNN 和 3D 卷积神经网络，并提出了一个采用时间卷积的注意力生成网络，用于在帧之间提取时间信息。在 MARS 数据集上进行评估，并通过一大部分超越了最新技术的方法。