We present PVO, a novel panoptic visual odometry framework to achieve more
comprehensive modeling of the scene motion, geometry, and panoptic segmentation
information. Our PVO models visual odometry (VO) and video panoptic
segmentation (VPS) in a unified view, which makes the two tasks mutually
beneficial. Specifically, we introduce a panoptic update module into the VO
Module with the guidance of image panoptic segmentation. This Panoptic-Enhanced
VO Module can alleviate the impact of dynamic objects in the camera pose
estimation with a panoptic-aware dynamic mask. On the other hand, the
VO-Enhanced VPS Module also improves the segmentation accuracy by fusing the
panoptic segmentation result of the current frame on the fly to the adjacent
frames, using geometric information such as camera pose, depth, and optical
flow obtained from the VO Module. These two modules contribute to each other
through recurrent iterative optimization. Extensive experiments demonstrate
that PVO outperforms state-of-the-art methods in both visual odometry and video
panoptic segmentation tasks.

PVO 是一种新型全景视觉里程计框架，它采用图像全景分割指导增强视觉里程计模块，并且通过迭代优化实现全景视觉里程计和视频全景分割的相互促进。实验证明，与现有方法相比，PVO 在视觉里程计和视频全景分割任务方面的性能更优。