Self-supervised learning (SSL) has proven effective in learning high-quality
representations for various downstream tasks, with a primary focus on semantic
tasks. However, its application in geometric tasks remains underexplored,
partially due to the absence of a standardized evaluation method for geometric
representations. To address this gap, we introduce a new pose-estimation
benchmark for assessing SSL geometric representations, which demands training
without semantic or pose labels and achieving proficiency in both semantic and
geometric downstream tasks. On this benchmark, we study enhancing SSL geometric
representations without sacrificing semantic classification accuracy. We find
that leveraging mid-layer representations improves pose-estimation performance
by 10-20%. Further, we introduce an unsupervised trajectory-regularization
loss, which improves performance by an additional 4% and improves
generalization ability on out-of-distribution data. We hope the proposed
benchmark and methods offer new insights and improvements in self-supervised
geometric representation learning.

自我监督学习在几何任务中的应用相对较少，本研究提出了一种评估自我监督几何表示方法的新姿态估计基准，通过在无语义或姿态标签的情况下训练，并在语义和几何下游任务上取得熟练度来研究强化自我监督几何表示方法，结果表明利用中间层表示可以提高姿态估计性能 10-20%，进一步引入无监督轨迹正则化损失可以额外提高 4% 的性能并改善对分布外数据的泛化能力，希望提出的基准和方法能为自我监督几何表示学习提供新的见解和改进。

轨迹规范化增强自监督几何表征

Trajectory Regularization Enhances Self-Supervised Geometric  Representation

Due to recent advances in pose-estimation methods, human motion can be
extracted from a common video in the form of 3D skeleton sequences. Despite
wonderful application opportunities, effective and efficient content-based
access to large volumes of such spatio-temporal skeleton data still remains a
challenging problem. In this paper, we propose a novel content-based
text-to-motion retrieval task, which aims at retrieving relevant motions based
on a specified natural-language textual description. To define baselines for
this uncharted task, we employ the BERT and CLIP language representations to
encode the text modality and successful spatio-temporal models to encode the
motion modality. We additionally introduce our transformer-based approach,
called Motion Transformer (MoT), which employs divided space-time attention to
effectively aggregate the different skeleton joints in space and time. Inspired
by the recent progress in text-to-image/video matching, we experiment with two
widely-adopted metric-learning loss functions. Finally, we set up a common
evaluation protocol by defining qualitative metrics for assessing the quality
of the retrieved motions, targeting the two recently-introduced KIT
Motion-Language and HumanML3D datasets. The code for reproducing our results is
available at this https URL

本文探讨如何实现基于文本描述的运动检索任务，利用姿态估计、文本编码和基于分割空时注意力的 Motion Transformer 模型，对大量 3D 骨骼序列进行内容检索，实现了相应的定量度量评估。