Watching a 360{\deg} sports video requires a viewer to continuously select a viewing angle, either through a sequence of mouse clicks or head movements. To relieve the viewer from this "360 piloting" task, we propose "deep 360 pilot" -- a deep learning-based agent for piloting through 360{\deg} sports videos automatically. At each frame, the agent observes a panoramic image and has the knowledge of previously selected viewing angles. The task of the agent is to shift the current viewing angle (i.e. action) to the next preferred one (i.e., goal). We propose to directly learn an online policy of the agent from data. We use the policy gradient technique to jointly train our pipeline: by minimizing (1) a regression loss measuring the distance between the selected and ground truth viewing angles, (2) a smoothness loss encouraging smooth transition in viewing angle, and (3) maximizing an expected reward of focusing on a foreground object. To evaluate our method, we build a new 360-Sports video dataset consisting of five sports domains. We train domain-specific agents and achieve the best performance on viewing angle selection accuracy and transition smoothness compared to [51] and other baselines.

提出了一种基于深度学习的360度运动视频智能导航器，通过直接从数据中学习在线策略，使用策略梯度技术联合训练管道，包括最小化测量选择和实际观看角度之间距离的回归损失、平滑过渡损失和最大化关注前景对象的预期奖励等。通过对五个领域具体运动的培训，产生了最佳的表现，并在选择角度准确性和过渡平稳性方面超越其他基线。

Deep 360 Pilot：学习一种深度智能代理，用于 360° 体育视频的飞行驾驶