Identifying individuals in unconstrained video settings is a valuable yet
challenging task in biometric analysis due to variations in appearances,
environments, degradations, and occlusions. In this paper, we present ShARc, a
multimodal approach for video-based person identification in uncontrolled
environments that emphasizes 3-D body shape, pose, and appearance. We introduce
two encoders: a Pose and Shape Encoder (PSE) and an Aggregated Appearance
Encoder (AAE). PSE encodes the body shape via binarized silhouettes, skeleton
motions, and 3-D body shape, while AAE provides two levels of temporal
appearance feature aggregation: attention-based feature aggregation and
averaging aggregation. For attention-based feature aggregation, we employ
spatial and temporal attention to focus on key areas for person distinction.
For averaging aggregation, we introduce a novel flattening layer after
averaging to extract more distinguishable information and reduce overfitting of
attention. We utilize centroid feature averaging for gallery registration. We
demonstrate significant improvements over existing state-of-the-art methods on
public datasets, including CCVID, MEVID, and BRIAR.

在本篇论文中，我们介绍了一种名为 ShARc 的多模态方法，用于在非受控环境中基于视频进行人员识别，着重考虑了 3D 身体形状、姿势和外貌。我们引入了两个编码器：姿势和形状编码器（PSE）和聚合外貌编码器（AAE）。PSE 通过二值化轮廓、骨骼运动和 3D 身体形状对身体形状进行编码，而 AAE 提供了两个级别的时间外貌特征聚合：基于注意力的特征聚合和平均聚合。我们展示了在公共数据集，包括 CCVID、MEVID 和 BRIAR 上，相对于现有最先进方法的显著改进。