Existing methods detect the keypoints in a non-differentiable way, therefore
they can not directly optimize the position of keypoints through
back-propagation. To address this issue, we present a partially differentiable
keypoint detection module, which outputs accurate sub-pixel keypoints. The
reprojection loss is then proposed to directly optimize these sub-pixel
keypoints, and the dispersity peak loss is presented for accurate keypoints
regularization. We also extract the descriptors in a sub-pixel way, and they
are trained with the stable neural reprojection error loss. Moreover, a
lightweight network is designed for keypoint detection and descriptor
extraction, which can run at 95 frames per second for 640x480 images on a
commercial GPU. On homography estimation, camera pose estimation, and visual
(re-)localization tasks, the proposed method achieves equivalent performance
with the state-of-the-art approaches, while greatly reduces the inference time.

提出了一种局部可微的关键点检测模块，能够输出精确的亚像素关键点，用于在各种任务中达到与现有最先进方法相当的性能，同时大大减少了推理时间。