Over the last decades, ample achievements have been made on Structure from
motion (SfM). However, the vast majority of them basically work in an offline
manner, i.e., images are firstly captured and then fed together into a SfM
pipeline for obtaining poses and sparse point cloud. In this work, on the
contrary, we present an on-the-fly SfM: running online SfM while image
capturing, the newly taken On-the-Fly image is online estimated with the
corresponding pose and points, i.e., what you capture is what you get.
Specifically, our approach firstly employs a vocabulary tree that is
unsupervised trained using learning-based global features for fast image
retrieval of newly fly-in image. Then, a robust feature matching mechanism with
least squares (LSM) is presented to improve image registration performance.
Finally, via investigating the influence of newly fly-in image's connected
neighboring images, an efficient hierarchical weighted local bundle adjustment
(BA) is used for optimization. Extensive experimental results demonstrate that
on-the-fly SfM can meet the goal of robustly registering the images while
capturing in an online way.

通过使用词汇树对新拍摄的图像进行快速检索，然后采用最小二乘法进行鲁棒特征匹配，并通过优化实现高效的分层加权局部捆绑调整，实现了在线拍摄时图像的实时注册。