Researchers have attempted utilizing deep neural network (DNN) to learn novel
local features from images inspired by its recent successes on a variety of
vision tasks. However, existing DNN-based algorithms have not achieved such
remarkable progress that could be partly attributed to insufficient utilization
of the interactive characters between local feature detector and descriptor. To
alleviate these difficulties, we emphasize two desired properties, i.e.,
repeatability and reliability, to simultaneously summarize the inherent and
interactive characters of local feature detector and descriptor. Guided by
these properties, a self-supervised framework, namely self-evolving keypoint
detection and description (SEKD), is proposed to learn an advanced local
feature model from unlabeled natural images. Additionally, to have performance
guarantees, novel training strategies have also been dedicatedly designed to
minimize the gap between the learned feature and its properties. We benchmark
the proposed method on homography estimation, relative pose estimation, and
structure-from-motion tasks. Extensive experimental results demonstrate that
the proposed method outperforms popular hand-crafted and DNN-based methods by
remarkable margins. Ablation studies also verify the effectiveness of each
critical training strategy. We will release our code along with the trained
model publicly.

使用自监督框架 SEKD 从未标记的自然图像中学习高级局部特征模型，实现了与传统手工精制和基于深度神经网络方法相比表现更好的齐次估计、相对姿态估计和结构运动任务检测。