It is especially challenging to achieve real-time human motion tracking on a
standalone VR Head-Mounted Display (HMD) such as Meta Quest and PICO. In this
paper, we propose HMD-Poser, the first unified approach to recover full-body
motions using scalable sparse observations from HMD and body-worn IMUs. In
particular, it can support a variety of input scenarios, such as HMD,
HMD+2IMUs, HMD+3IMUs, etc. The scalability of inputs may accommodate users'
choices for both high tracking accuracy and easy-to-wear. A lightweight
temporal-spatial feature learning network is proposed in HMD-Poser to guarantee
that the model runs in real-time on HMDs. Furthermore, HMD-Poser presents
online body shape estimation to improve the position accuracy of body joints.
Extensive experimental results on the challenging AMASS dataset show that
HMD-Poser achieves new state-of-the-art results in both accuracy and real-time
performance. We also build a new free-dancing motion dataset to evaluate
HMD-Poser's on-device performance and investigate the performance gap between
synthetic data and real-captured sensor data. Finally, we demonstrate our
HMD-Poser with a real-time Avatar-driving application on a commercial HMD. Our
code and free-dancing motion dataset are available
this https URL

使用 HMD-Poser 方法实现了基于 VR 头戴式显示器的人体实时运动追踪，通过 HMD 和穿戴式 IMUs 之间的观察数据，提高了追踪精确度和易于佩戴性。通过实验表明，HMD-Poser 在精确度和实时性方面达到了最新的最佳效果，并通过实时的 Avatar-driving 应用程序证明了其性能。