This paper proposes a novel application system for the generation of
three-dimensional (3D) character animation driven by markerless human body
motion capturing. The entire pipeline of the system consists of five stages: 1)
the capturing of motion data using multiple cameras, 2) detection of the
two-dimensional (2D) human body joints, 3) estimation of the 3D joints, 4)
calculation of bone transformation matrices, and 5) generation of character
animation. The main objective of this study is to generate a 3D skeleton and
animation for 3D characters using multi-view images captured by ordinary
cameras. The computational complexity of the 3D skeleton reconstruction based
on 3D vision has been reduced as needed to achieve frame-by-frame motion
capturing. The experimental results reveal that our system can effectively and
efficiently capture human actions and use them to animate 3D cartoon characters
in real-time.

本文描述了一种基于无标记人体动作捕获的三维角色动画生成系统，采用视角多样的相机捕捉人体动作信息，并通过多种技术手段计算出骨骼变换状况，从而能够高效准确地实现三维骨骼重建与实时动画生成。

基于多视角摄像机的无标记身体运动捕捉技术用于 3D 角色动画制作

Markerless Body Motion Capturing for 3D Character Animation based on Multi-view Cameras

The increasing availability of video recordings made by multiple cameras has
offered new means for mitigating occlusion and depth ambiguities in pose and
motion reconstruction methods. Yet, multi-view algorithms strongly depend on
camera parameters; particularly, the relative transformations between the
cameras. Such a dependency becomes a hurdle once shifting to dynamic capture in
uncontrolled settings. We introduce FLEX (Free muLti-view rEconstruXion), an
end-to-end extrinsic parameter-free multi-view model. FLEX is extrinsic
parameter-free (dubbed ep-free) in the sense that it does not require extrinsic
camera parameters. Our key idea is that the 3D angles between skeletal parts,
as well as bone lengths, are invariant to the camera position. Hence, learning
3D rotations and bone lengths rather than locations allows predicting common
values for all camera views. Our network takes multiple video streams, learns
fused deep features through a novel multi-view fusion layer, and reconstructs a
single consistent skeleton with temporally coherent joint rotations. We
demonstrate quantitative and qualitative results on three public datasets, and
on synthetic multi-person video streams captured by dynamic cameras. We compare
our model to state-of-the-art methods that are not ep-free and show that in the
absence of camera parameters, we outperform them by a large margin while
obtaining comparable results when camera parameters are available. Code,
trained models, and other materials are available on our project page.

本研究提出了一种不需要相机参数的多视角模型 FLEX，通过学习骨架部位的 3D 旋转和长度，该模型能够重构一致的骨架，在未知相机设置情况下能够取得比现有技术更好的结果。