Full-body avatar presence is crucial for immersive social and environmental
interactions in digital reality. However, current devices only provide three
six degrees of freedom (DOF) poses from the headset and two controllers (i.e.
three-point trackers). Because it is a highly under-constrained problem,
inferring full-body pose from these inputs is challenging, especially when
supporting the full range of body proportions and use cases represented by the
general population. In this paper, we propose a deep learning framework,
DivaTrack, which outperforms existing methods when applied to diverse body
sizes and activities. We augment the sparse three-point inputs with linear
accelerations from Inertial Measurement Units (IMU) to improve foot contact
prediction. We then condition the otherwise ambiguous lower-body pose with the
predictions of foot contact and upper-body pose in a two-stage model. We
further stabilize the inferred full-body pose in a wide range of configurations
by learning to blend predictions that are computed in two reference frames,
each of which is designed for different types of motions. We demonstrate the
effectiveness of our design on a large dataset that captures 22 subjects
performing challenging locomotion for three-point tracking, including lunges,
hula-hooping, and sitting. As shown in a live demo using the Meta VR headset
and Xsens IMUs, our method runs in real-time while accurately tracking a user's
motion when they perform a diverse set of movements.

基于深度学习框架的 DivaTrack 能够通过增加稀疏三点输入与输入的线性加速度来改善脚接触预测，利用两阶段模型将不明确的下半身姿势与脚接触和上半身姿势的预测相结合，通过在不同类型运动的两个参考坐标系中学习预测进行平滑，能够在真实时间内精确追踪用户在多种动作中的运动。