Recent advances in computer vision have made significant progress in tracking
and pose estimation of sports players. However, there have been fewer studies
on behavior prediction with pose estimation in sports, in particular, the
prediction of soccer fouls is challenging because of the smaller image size of
each player and of difficulty in the usage of e.g., the ball and pose
information. In our research, we introduce an innovative deep learning approach
for anticipating soccer fouls. This method integrates video data, bounding box
positions, image details, and pose information by curating a novel soccer foul
dataset. Our model utilizes a combination of convolutional and recurrent neural
networks (CNNs and RNNs) to effectively merge information from these four
modalities. The experimental results show that our full model outperformed the
ablated models, and all of the RNN modules, bounding box position and image,
and estimated pose were useful for the foul prediction. Our findings have
important implications for a deeper understanding of foul play in soccer and
provide a valuable reference for future research and practice in this area.

我们的研究引入了一种创新的深度学习方法，用于预测足球犯规。该方法通过构建一个新颖的足球犯规数据集，将视频数据、边界框位置、图像细节和姿势信息相结合，利用卷积和循环神经网络 (CNNs 和 RNNs) 从这四个模态中有效地融合信息。实验结果表明我们的完整模型优于其他模型，所有的 RNN 模块、边界框位置、图像和估计的姿势在犯规预测中都有用。我们的研究结果对于更深入地理解足球比赛中的犯规行为具有重要意义，并为该领域的未来研究和实践提供了有价值的参考。