In this work, we present a methodology to shape a fisheye-specific
representation space that reflects the interaction between distortion and
semantic context present in this data modality. Fisheye data has the wider
field of view advantage over other types of cameras, but this comes at the
expense of high radial distortion. As a result, objects further from the center
exhibit deformations that make it difficult for a model to identify their
semantic context. While previous work has attempted architectural and training
augmentation changes to alleviate this effect, no work has attempted to guide
the model towards learning a representation space that reflects this
interaction between distortion and semantic context inherent to fisheye data.
We introduce an approach to exploit this relationship by first extracting
distortion class labels based on an object's distance from the center of the
image. We then shape a backbone's representation space with a weighted
contrastive loss that constrains objects of the same semantic class and
distortion class to be close to each other within a lower dimensional embedding
space. This backbone trained with both semantic and distortion information is
then fine-tuned within an object detection setting to empirically evaluate the
quality of the learnt representation. We show this method leads to performance
improvements by as much as 1.1% mean average precision over standard object
detection strategies and .6% improvement over other state of the art
representation learning approaches.

本文提出一种方法来塑造一个鱼眼特定的表示空间，以反映这种数据模态中存在的畸变和语义上下文之间的相互作用，该方法向模型提供了关于畸变和语义上下文之间相互作用的学习方法，并应用于目标检测任务中，从而在平均精度上实现了 1.1% 的性能提升，并超过了其他现有的表征学习方法 0.6% 的 性能。

利用鱼眼数据中的畸变 - 语义交互

Exploiting the Distortion-Semantic Interaction in Fisheye Data

Wide-angle lenses are commonly used in perception tasks requiring a large
field of view. Unfortunately, these lenses produce significant distortions
making conventional models that ignore the distortion effects unable to adapt
to wide-angle images. In this paper, we present a novel transformer-based model
that automatically adapts to the distortion produced by wide-angle lenses. We
leverage the physical characteristics of such lenses, which are analytically
defined by the radial distortion profile (assumed to be known), to develop a
distortion aware radial swin transformer (DarSwin). In contrast to conventional
transformer-based architectures, DarSwin comprises a radial patch partitioning,
a distortion-based sampling technique for creating token embeddings, and a
polar position encoding for radial patch merging. We validate our method on
classification tasks using synthetically distorted ImageNet data and show
through extensive experiments that DarSwin can perform zero-shot adaptation to
unseen distortions of different wide-angle lenses. Compared to other baselines,
DarSwin achieves the best results (in terms of Top-1 and -5 accuracy), when
tested on in-distribution data, with almost 2% (6%) gain in Top-1 accuracy
under medium (high) distortion levels, and comparable to the state-of-the-art
under low and very low distortion levels (perspective-like images).

本文提出了一种基于变压器的新型模型 DarSwin，可以自适应宽角镜头所产生的畸变，并对各种畸变进行了零样本适应。与其他基线相比，DarSwin 在中度和高度畸变水平下获得了最佳结果，并且在低度和非常低度的畸变水平下与现有技术相媲美。

DarSwin: 扭曲感知径向 Swin Transformer

DarSwin: Distortion Aware Radial Swin Transformer

Self-calibration of camera intrinsics and radial distortion has a long
history of research in the computer vision community. However, it remains rare
to see real applications of such techniques to modern Simultaneous Localization
And Mapping (SLAM) systems, especially in driving scenarios. In this paper, we
revisit the geometric approach to this problem, and provide a theoretical proof
that explicitly shows the ambiguity between radial distortion and scene depth
when two-view geometry is used to self-calibrate the radial distortion. In view
of such geometric degeneracy, we propose a learning approach that trains a
convolutional neural network (CNN) on a large amount of synthetic data. We
demonstrate the utility of our proposed method by applying it as a
checkerboard-free calibration tool for SLAM, achieving comparable or superior
performance to previous learning and hand-crafted methods.

本文提出了一种学习方法，使用卷积神经网络对相机自校准进行校正，以解决使用双视图几何方法自校准径向畸变时径向畸变和场景深度之间的歧义，该方法在 SLAM 应用中表现出与以前学习和手工方法相当或优异的性能。