Leveraging wearable devices for motion reconstruction has emerged as an
economical and viable technique. Certain methodologies employ sparse Inertial
Measurement Units (IMUs) on the human body and harness data-driven strategies
to model human poses. However, the reconstruction of motion based solely on
sparse IMUs data is inherently fraught with ambiguity, a consequence of
numerous identical IMU readings corresponding to different poses. In this
paper, we explore the spatial importance of multiple sensors, supervised by
text that describes specific actions. Specifically, uncertainty is introduced
to derive weighted features for each IMU. We also design a Hierarchical
Temporal Transformer (HTT) and apply contrastive learning to achieve precise
temporal and feature alignment of sensor data with textual semantics.
Experimental results demonstrate our proposed approach achieves significant
improvements in multiple metrics compared to existing methods. Notably, with
textual supervision, our method not only differentiates between ambiguous
actions such as sitting and standing but also produces more precise and natural
motion.

利用可穿戴设备进行动作重建已成为一种经济且可行的技术，在稀疏的惯性测量单元（IMUs）数据上建立人体姿势模型存在着困扰，本文通过多传感器的空间重要性和文本描述的监督引入不确定性来获取每个 IMU 的加权特征，并设计了一种层次时间变换器（HTT）和对比学习来实现传感器数据与文本语义的精确时间和特征对齐。实验证明我们的方法在多个指标上相比现有方法有显著改进，尤其是在文本监督下，我们的方法不仅能区分诸如坐下和站起之类的模糊动作，还能产生更精确和自然的动作。