This paper introduces a simple yet effective strategy for exercise
classification and muscle group activation prediction (MGAP). These tasks have
significant implications for personal fitness, facilitating more affordable,
accessible, safer, and simpler exercise routines. This is particularly relevant
for novices and individuals with disabilities. Previous research in the field
is mostly dominated by the reliance on mounted sensors and a limited scope of
exercises, reducing practicality for everyday use. Furthermore, existing MGAP
methodologies suffer from a similar dependency on sensors and a restricted
range of muscle groups, often excluding strength training exercises, which are
pivotal for a comprehensive fitness regimen. Addressing these limitations, our
research employs a video-based deep learning framework that encompasses a broad
spectrum of exercises and muscle groups, including those vital for strength
training. Utilizing the "Workout/Exercises Video" dataset, our approach
integrates the X3D and SlowFast video activity recognition models in an
effective way to enhance exercise classification and MGAP performance. Our
findings demonstrate that this hybrid method obtained via weighted ensemble
outperforms existing baseline models in accuracy. Pretrained models play a
crucial role in enhancing overall performance, with optimal channel reduction
values for the SlowFast model identified near 10. Through an ablation study
that explores fine-tuning, we further elucidate the interrelation between the
two tasks. Our composite model, a weighted-average ensemble of X3D and
SlowFast, sets a new benchmark in both exercise classification and MGAP across
all evaluated categories, offering a robust solution to the limitations of
previous approaches.

通过使用基于视频的深度学习框架，整合 X3D 和 SlowFast 视频活动识别模型的加权融合方法，本研究在锻炼分类和肌肉群激活预测方面取得了显著的性能提高。

基于混合 X3D-SlowFast 网络的视频运动分类和激活肌群预测

Video-based Exercise Classification and Activated Muscle Group  Prediction with Hybrid X3D-SlowFast Network

Wearable sensors such as Inertial Measurement Units (IMUs) are often used to
assess the performance of human exercise. Common approaches use handcrafted
features based on domain expertise or automatically extracted features using
time series analysis. Multiple sensors are required to achieve high
classification accuracy, which is not very practical. These sensors require
calibration and synchronization and may lead to discomfort over longer time
periods. Recent work utilizing computer vision techniques has shown similar
performance using video, without the need for manual feature engineering, and
avoiding some pitfalls such as sensor calibration and placement on the body. In
this paper, we compare the performance of IMUs to a video-based approach for
human exercise classification on two real-world datasets consisting of Military
Press and Rowing exercises. We compare the performance using a single camera
that captures video in the frontal view versus using 5 IMUs placed on different
parts of the body. We observe that an approach based on a single camera can
outperform a single IMU by 10 percentage points on average. Additionally, a
minimum of 3 IMUs are required to outperform a single camera. We observe that
working with the raw data using multivariate time series classifiers
outperforms traditional approaches based on handcrafted or automatically
extracted features. Finally, we show that an ensemble model combining the data
from a single camera with a single IMU outperforms either data modality. Our
work opens up new and more realistic avenues for this application, where a
video captured using a readily available smartphone camera, combined with a
single sensor, can be used for effective human exercise classification.

本文比较了基于惯性测量单元 (IMU) 和基于视频的方法在军事按压和划船运动的人体运动分类中的表现，发现单个摄像头能比单个 IMU 提高 10 个百分点的分类准确率，而至少需要 3 个 IMU 才能超越单个摄像头。同时，使用基于多变量时间序列分类器进行原始数据处理的方法优于基于手工特征或自动提取特征的传统方法。最后，将单个摄像头和单个 IMU 的数据组合起来能超越任一数据模态，为使用智能手机摄像头和单一传感器进行有效的人体运动分类开辟了新的、更现实的途径。