Classification and localization of driving actions over time is important for
advanced driver-assistance systems and naturalistic driving studies. Temporal
localization is challenging because it requires robustness, reliability, and
accuracy. In this study, we aim to improve the temporal localization and
classification accuracy performance by adapting video action recognition and 2D
human-pose estimation networks to one model. Therefore, we design a
transformer-based fusion architecture to effectively combine 2D-pose features
and spatio-temporal features. The model uses 2D-pose features as the positional
embedding of the transformer architecture and spatio-temporal features as the
main input to the encoder of the transformer. The proposed solution is generic
and independent of the camera numbers and positions, giving frame-based class
probabilities as output. Finally, the post-processing step combines information
from different camera views to obtain final predictions and eliminate false
positives. The model performs well on the A2 test set of the 2023 NVIDIA AI
City Challenge for naturalistic driving action recognition, achieving the
overlap score of the organizer-defined distracted driver behaviour metric of
0.5079.

该研究旨在通过将视频动作识别和 2D 人体姿势估计网络结合为一个模型，改善时间定位和分类准确性表现。最终，通过从不同的摄像头视角获取信息并消除误报，该模型在 2023 年 NVIDIA AI 城市挑战的自然驾驶行为识别中表现出色，实现了优化的区域覆盖得分 0.5079。