Accurate 3D kinematics estimation of human body is crucial in various
applications for human health and mobility, such as rehabilitation, injury
prevention, and diagnosis, as it helps to understand the biomechanical loading
experienced during movement. Conventional marker-based motion capture is
expensive in terms of financial investment, time, and the expertise required.
Moreover, due to the scarcity of datasets with accurate annotations, existing
markerless motion capture methods suffer from challenges including unreliable
2D keypoint detection, limited anatomic accuracy, and low generalization
capability. In this work, we propose a novel biomechanics-aware network that
directly outputs 3D kinematics from two input views with consideration of
biomechanical prior and spatio-temporal information. To train the model, we
create synthetic dataset ODAH with accurate kinematics annotations generated by
aligning the body mesh from the SMPL-X model and a full-body OpenSim skeletal
model. Our extensive experiments demonstrate that the proposed approach, only
trained on synthetic data, outperforms previous state-of-the-art methods when
evaluated across multiple datasets, revealing a promising direction for
enhancing video-based human motion capture.

提出了一种基于生物力学的新型网络，利用两个输入视图直接输出三维运动学，通过合成数据集 ODAH 进行训练，实验证明该方法在多个数据集上超过了现有先进方法，为提升基于视频的人体动作捕捉提供了有前景的方向。

基于生物力学模型和合成训练数据的视频中的三维运动学估计

3D Kinematics Estimation from Video with a Biomechanical Model and  Synthetic Training Data

In this work, we investigate the influence of labeling methods on the
classification of human movements on data recorded using a marker-based motion
capture system. The dataset is labeled using two different approaches, one
based on video data of the movements, the other based on the movement
trajectories recorded using the motion capture system. The dataset is labeled
using two different approaches, one based on video data of the movements, the
other based on the movement trajectories recorded using the motion capture
system. The data was recorded from one participant performing a stacking
scenario comprising simple arm movements at three different speeds (slow,
normal, fast). Machine learning algorithms that include k-Nearest Neighbor,
Random Forest, Extreme Gradient Boosting classifier, Convolutional Neural
networks (CNN), Long Short-Term Memory networks (LSTM), and a combination of
CNN-LSTM networks are compared on their performance in recognition of these arm
movements. The models were trained on actions performed on slow and normal
speed movements segments and generalized on actions consisting of fast-paced
human movement. It was observed that all the models trained on normal-paced
data labeled using trajectories have almost 20% improvement in accuracy on test
data in comparison to the models trained on data labeled using videos of the
performed experiments.

本研究探讨了在使用基于标记的动作捕捉系统记录的数据中，标注方法对于人体动作分类的影响，并比较了基于视频标注和运动轨迹标注方式的监督机器学习算法在识别不同速度人体动作时的表现，结果表明，基于运动轨迹标注的模型在识别中速和正常速度人体动作时相对于基于视频标注的模型有着 20％左右的准确率提高。