Today's Mixed Reality head-mounted displays track the user's head pose in
world space as well as the user's hands for interaction in both Augmented
Reality and Virtual Reality scenarios. While this is adequate to support user
input, it unfortunately limits users' virtual representations to just their
upper bodies. Current systems thus resort to floating avatars, whose limitation
is particularly evident in collaborative settings. To estimate full-body poses
from the sparse input sources, prior work has incorporated additional trackers
and sensors at the pelvis or lower body, which increases setup complexity and
limits practical application in mobile settings. In this paper, we present
AvatarPoser, the first learning-based method that predicts full-body poses in
world coordinates using only motion input from the user's head and hands. Our
method builds on a Transformer encoder to extract deep features from the input
signals and decouples global motion from the learned local joint orientations
to guide pose estimation. To obtain accurate full-body motions that resemble
motion capture animations, we refine the arm joints' positions using an
optimization routine with inverse kinematics to match the original tracking
input. In our evaluation, AvatarPoser achieved new state-of-the-art results in
evaluations on large motion capture datasets (AMASS). At the same time, our
method's inference speed supports real-time operation, providing a practical
interface to support holistic avatar control and representation for Metaverse
applications.

AvatarPoser 是第一个仅使用用户的头部和手部运动输入来预测世界坐标中全身姿势的基于学习的方法，具有较高的准确性和实时操作速度，并能够提供支持 Metaverse 应用的全面性化虚拟人物控制和表现的实用界面。

AvatarPoser: 从稀疏动作感知中获取关节化全身姿势跟踪

AvatarPoser: Articulated Full-Body Pose Tracking from Sparse Motion Sensing

Achieving the expansion of domestic demand and the economic internal
circulation requires balanced and coordinated support from multiple industries
(domains) such as consumption, education, entertainment, engineering
infrastructure, etc., which is indispensable for maintaining economic
development. Metaverse applications may help with this task and can make many
industries more interesting, more efficient, and provide a better user
experience. The first challenge is that metaverse application development
inevitably requires the support of various artificial intelligence (AI)
technologies such as natural language processing (NLP), knowledge graph (KG),
computer vision (CV), and machine learning (ML), etc. However, existing
metaverse application development lacks a lightweight AI technology framework.
This paper proposes a flexible metaverse AI technology framework metaAID that
aims to support language and semantic technologies in the development of
digital twins and virtual humans. The second challenge is that the development
process of metaverse applications involves both technical development tasks and
manual editing work, and often becomes a heavyweight multi-team collaboration
project, not to mention the development of metaverse applications in multiple
industries. Our framework summarizes common AI technologies and application
development templates with common functional modules and interfaces. Based on
this framework, we have designed 5 applications for 3 industries around the
expansion of domestic demand and economic internal circulation. Experimental
results show that our framework can support AI technologies when developing
metaverse applications in different industries.

提出了一个新的灵活的元宇宙人工智能技术框架 metaAID，该框架可支持多种人工智能技术在数字孪生和虚拟人类的开发中使用。开发元宇宙应用程序涉及技术开发任务和手动编辑工作，并经常成为一个重量级多团队协作项目，而多个行业的元宇宙应用程序开发更加困难。在此框架的基础上，我们设计了 5 个应用程序来支持国内需求和经济内部循环的 3 个行业，并通过实验结果表明，我们的框架可以支持各行业开发元宇宙应用时的人工智能技术。