Novel view synthesis from a single input image is a challenging task, where
the goal is to generate a new view of a scene from a desired camera pose that
may be separated by a large motion. The highly uncertain nature of this
synthesis task due to unobserved elements within the scene (i.e., occlusion)
and outside the field-of-view makes the use of generative models appealing to
capture the variety of possible outputs. In this paper, we propose a novel
generative model which is capable of producing a sequence of photorealistic
images consistent with a specified camera trajectory, and a single starting
image. Our approach is centred on an autoregressive conditional diffusion-based
model capable of interpolating visible scene elements, and extrapolating
unobserved regions in a view, in a geometrically consistent manner.
Conditioning is limited to an image capturing a single camera view and the
(relative) pose of the new camera view. To measure the consistency over a
sequence of generated views, we introduce a new metric, the thresholded
symmetric epipolar distance (TSED), to measure the number of consistent frame
pairs in a sequence. While previous methods have been shown to produce high
quality images and consistent semantics across pairs of views, we show
empirically with our metric that they are often inconsistent with the desired
camera poses. In contrast, we demonstrate that our method produces both
photorealistic and view-consistent imagery.

本文提出了一种基于条件扩散模型的新型生成模型，用于从单个输入图像生成相对的摄像机视图的连续序列的照片般逼真、视角一致、空间几何连续的图像。

具有扩散模型的长期光度一致的新视角综合

Long-Term Photometric Consistent Novel View Synthesis with Diffusion  Models

Warping-based video stabilizers smooth camera trajectory by constraining each
pixel's displacement and warp stabilized frames from unstable ones accordingly.
However, since the view outside the boundary is not available during warping,
the resulting holes around the boundary of the stabilized frame must be
discarded (i.e., cropping) to maintain visual consistency, and thus does leads
to a tradeoff between stability and cropping ratio. In this paper, we make a
first attempt to address this issue by proposing a new Out-of-boundary View
Synthesis (OVS) method. By the nature of spatial coherence between adjacent
frames and within each frame, OVS extrapolates the out-of-boundary view by
aligning adjacent frames to each reference one. Technically, it first
calculates the optical flow and propagates it to the outer boundary region
according to the affinity, and then warps pixels accordingly. OVS can be
integrated into existing warping-based stabilizers as a plug-and-play module to
significantly improve the cropping ratio of the stabilized results. In
addition, stability is improved because the jitter amplification effect caused
by cropping and resizing is reduced. Experimental results on the NUS benchmark
show that OVS can improve the performance of five representative
state-of-the-art methods in terms of objective metrics and subjective visual
quality. The code is publicly available at
this https URL

本文提出了一种基于 Out-of-boundary View Synthesis (OVS) 的新型视图插值方法，即在变形稳定的基础上，通过对相邻帧之间的空间一致性来推导出边界外视图，从而显著提高稳定性能和裁剪比率。

全帧视频稳定的出界视角合成

Out-of-boundary View Synthesis Towards Full-Frame Video Stabilization

We propose a novel dense mapping framework for sparse visual SLAM systems
which leverages a compact scene representation. State-of-the-art sparse visual
SLAM systems provide accurate and reliable estimates of the camera trajectory
and locations of landmarks. While these sparse maps are useful for
localization, they cannot be used for other tasks such as obstacle avoidance or
scene understanding. In this paper we propose a dense mapping framework to
complement sparse visual SLAM systems which takes as input the camera poses,
keyframes and sparse points produced by the SLAM system and predicts a dense
depth image for every keyframe. We build on CodeSLAM and use a variational
autoencoder (VAE) which is conditioned on intensity, sparse depth and
reprojection error images from sparse SLAM to predict an uncertainty-aware
dense depth map. The use of a VAE then enables us to refine the dense depth
images through multi-view optimization which improves the consistency of
overlapping frames. Our mapper runs in a separate thread in parallel to the
SLAM system in a loosely coupled manner. This flexible design allows for
integration with arbitrary metric sparse SLAM systems without delaying the main
SLAM process. Our dense mapper can be used not only for local mapping but also
globally consistent dense 3D reconstruction through TSDF fusion. We demonstrate
our system running with ORB-SLAM3 and show accurate dense depth estimation
which could enable applications such as robotics and augmented reality.

本篇文章提出了一种用于稀疏视觉 SLAM 系统的新型稠密建图框架，它基于一种紧凑的场景表示方式，可以用于机器人和增强现实等应用领域。该框架使用了变分自编码器，通过多视角优化可以改善重叠帧的一致性，对于任意度量的稀疏 SLAM 系统进行无缝集成，能够进行全局一致的稠密三维重建。