The rapid and precise localization and prediction of a ball are critical for
developing agile robots in ball sports, particularly in sports like tennis
characterized by high-speed ball movements and powerful spins. The Magnus
effect induced by spin adds complexity to trajectory prediction during flight
and bounce dynamics upon contact with the ground. In this study, we introduce
an innovative approach that combines a multi-camera system with factor graphs
for real-time and asynchronous 3D tennis ball localization. Additionally, we
estimate hidden states like velocity and spin for trajectory prediction.
Furthermore, to enhance spin inference early in the ball's flight, where
limited observations are available, we integrate human pose data using a
temporal convolutional network (TCN) to compute spin priors within the factor
graph. This refinement provides more accurate spin priors at the beginning of
the factor graph, leading to improved early-stage hidden state inference for
prediction. Our result shows the trained TCN can predict the spin priors with
RMSE of 5.27 Hz. Integrating TCN into the factor graph reduces the prediction
error of landing positions by over 63.6% compared to a baseline method that
utilized an adaptive extended Kalman filter.

通过多相机系统与因子图的创新方法，结合人体姿态数据和时间卷积网络（TCN）来定位网球、预测轨迹，估计速度和旋转，提供更准确的旋转先验信息，从而提高运动初始阶段的隐藏状态推理能力。