Image stitching is typically decomposed into three phases: registration,
which aligns the source images with a common target image; seam finding, which
determines for each target pixel the source image it should come from; and
blending, which smooths transitions over the seams. As described in [1], the
seam finding phase attempts to place seams between pixels where the transition
between source images is not noticeable. Here, we observe that the most
problematic failures of this approach occur when objects are cropped, omitted,
or duplicated. We therefore take an object-centered approach to the problem,
leveraging recent advances in object detection [2,3,4]. We penalize candidate
solutions with this class of error by modifying the energy function used in the
seam finding stage. This produces substantially more realistic stitching
results on challenging imagery. In addition, these methods can be used to
determine when there is non-recoverable occlusion in the input data, and also
suggest a simple evaluation metric that can be used to evaluate the output of
stitching algorithms.

本文针对图像拼接过程中目标被裁剪、遗漏或复制而导致的错误进行研究，提出一种基于目标检测的算法来解决这个问题，并将其用于修改拼接过程中的能量函数，从而获得更加逼真和鲁棒的拼接结果。同时，该方法还可以用于检测输入数据中不可恢复的遮挡，并提出了一种简单的评估指标来评估图像拼接算法的输出结果。

基于目标的图像拼接

Object-centered image stitching

Panorama creation is one of the most widely deployed techniques in computer
vision. In addition to industry applications such as Google Street View, it is
also used by millions of consumers in smartphones and other cameras.
Traditionally, the problem is decomposed into three phases: registration, which
picks a single transformation of each source image to align it to the other
inputs, seam finding, which selects a source image for each pixel in the final
result, and blending, which fixes minor visual artifacts. Here, we observe that
the use of a single registration often leads to errors, especially in scenes
with significant depth variation or object motion. We propose instead the use
of multiple registrations, permitting regions of the image at different depths
to be captured with greater accuracy. MRF inference techniques naturally extend
to seam finding over multiple registrations, and we show here that their energy
functions can be readily modified with new terms that discourage duplication
and tearing, common problems that are exacerbated by the use of multiple
registrations. Our techniques are closely related to layer-based stereo, and
move image stitching closer to explicit scene modeling. Experimental evidence
demonstrates that our techniques often generate significantly better panoramas
when there is substantial motion or parallax.

该论文提出了一种改进的全景图创建算法，采用多重图像配准的方法，提高了深度差异和物体运动场景下的图像拼接的精度，实验结果表明该算法在存在较大运动或视差时能够生成明显更好的全景图。