Keypoint detection and matching is a fundamental task in many computer vision
problems, from shape reconstruction, to structure from motion, to AR/VR
applications and robotics. It is a well-studied problem with remarkable
successes such as SIFT, and more recent deep learning approaches. While great
robustness is exhibited by these techniques with respect to noise, illumination
variation, and rigid motion transformations, less attention has been placed on
image distortion sensitivity. In this work, we focus on the case when this is
caused by the geometry of the cameras used for image acquisition, and consider
the keypoint detection and matching problem between the hybrid scenario of a
fisheye and a projective image. We build on a state-of-the-art approach and
derive a self-supervised procedure that enables training an interest point
detector and descriptor network. We also collected two new datasets for
additional training and testing in this unexplored scenario, and we demonstrate
that current approaches are suboptimal because they are designed to work in
traditional projective conditions, while the proposed approach turns out to be
the most effective.

本文通过自监督学习的方法针对鱼眼和投影图像混合的情况，研究关键点检测和匹配问题，提出了一种训练兴趣点检测器和描述符网络的方法，并证明了目前的方法不是最优解，提出的方法是最有效的。

鱼眼和透视图像自监督兴趣点检测和描述

Self-supervised Interest Point Detection and Description for Fisheye and  Perspective Images

In this work, we present a methodology to shape a fisheye-specific
representation space that reflects the interaction between distortion and
semantic context present in this data modality. Fisheye data has the wider
field of view advantage over other types of cameras, but this comes at the
expense of high radial distortion. As a result, objects further from the center
exhibit deformations that make it difficult for a model to identify their
semantic context. While previous work has attempted architectural and training
augmentation changes to alleviate this effect, no work has attempted to guide
the model towards learning a representation space that reflects this
interaction between distortion and semantic context inherent to fisheye data.
We introduce an approach to exploit this relationship by first extracting
distortion class labels based on an object's distance from the center of the
image. We then shape a backbone's representation space with a weighted
contrastive loss that constrains objects of the same semantic class and
distortion class to be close to each other within a lower dimensional embedding
space. This backbone trained with both semantic and distortion information is
then fine-tuned within an object detection setting to empirically evaluate the
quality of the learnt representation. We show this method leads to performance
improvements by as much as 1.1% mean average precision over standard object
detection strategies and .6% improvement over other state of the art
representation learning approaches.

本文提出一种方法来塑造一个鱼眼特定的表示空间，以反映这种数据模态中存在的畸变和语义上下文之间的相互作用，该方法向模型提供了关于畸变和语义上下文之间相互作用的学习方法，并应用于目标检测任务中，从而在平均精度上实现了 1.1% 的性能提升，并超过了其他现有的表征学习方法 0.6% 的 性能。

利用鱼眼数据中的畸变 - 语义交互

Exploiting the Distortion-Semantic Interaction in Fisheye Data

We present the WoodScape fisheye semantic segmentation challenge for
autonomous driving which was held as part of the CVPR 2021 Workshop on
Omnidirectional Computer Vision (OmniCV). This challenge is one of the first
opportunities for the research community to evaluate the semantic segmentation
techniques targeted for fisheye camera perception. Due to strong radial
distortion standard models don't generalize well to fisheye images and hence
the deformations in the visual appearance of objects and entities needs to be
encoded implicitly or as explicit knowledge. This challenge served as a medium
to investigate the challenges and new methodologies to handle the complexities
with perception on fisheye images. The challenge was hosted on CodaLab and used
the recently released WoodScape dataset comprising of 10k samples. In this
paper, we provide a summary of the competition which attracted the
participation of 71 global teams and a total of 395 submissions. The top teams
recorded significantly improved mean IoU and accuracy scores over the baseline
PSPNet with ResNet-50 backbone. We summarize the methods of winning algorithms
and analyze the failure cases. We conclude by providing future directions for
the research.

本文介绍了 WoodScape 鱼眼语义分割挑战赛，它是 CVPR 2021 奥姆尼方向计算机视觉研讨会的一部分，旨在评估用于鱼眼相机感知的语义分割技术。在挑战赛中，来自世界各地的 71 个团队共提交了 395 份报告，获胜算法取得了比基准更高的平均 IoU 和准确度分数，并分析了失败案例以及未来的研究方向。