We propose a novel method, LoLep, which regresses Locally-Learned planes from
a single RGB image to represent scenes accurately, thus generating better novel
views. Without the depth information, regressing appropriate plane locations is
a challenging problem. To solve this issue, we pre-partition the disparity
space into bins and design a disparity sampler to regress local offsets for
multiple planes in each bin. However, only using such a sampler makes the
network not convergent; we further propose two optimizing strategies that
combine with different disparity distributions of datasets and propose an
occlusion-aware reprojection loss as a simple yet effective geometric
supervision technique. We also introduce a self-attention mechanism to improve
occlusion inference and present a Block-Sampling Self-Attention (BS-SA) module
to address the problem of applying self-attention to large feature maps. We
demonstrate the effectiveness of our approach and generate state-of-the-art
results on different datasets. Compared to MINE, our approach has an LPIPS
reduction of 4.8%-9.0% and an RV reduction of 83.1%-84.7%. We also evaluate the
performance on real-world images and demonstrate the benefits.

我们提出了一种新的方法 LoLep，通过从单个 RGB 图像回归学习本地平面，准确地表示场景，从而生成更好的新视图。该方法将深度信息从中剔除，且在无深度信息的情况下回归合适的平面位置是一个具有挑战性的问题。为了解决这个问题，我们将视差空间预划分为不同的区间，并设计了一个视差采样器用于在每个区间中回归多个平面的局部偏移量。然而，仅使用这样的采样器会导致网络无法收敛；因此我们进一步提出了两种优化策略，将其与不同数据集的视差分布相结合，并提出了一种考虑遮挡的再投影损失作为一种简单有效的几何监督技术。我们还引入了一种自注意机制以改善遮挡推断，并提出了一个块采样自注意力模块（BS-SA）来解决将自注意力应用于大型特征图的问题。我们证明了我们的方法的有效性，并在不同数据集上生成了最先进的结果。与 MINE 相比，我们的方法在 LPIPS 方面有 4.8%-9.0% 的减少和 83.1%-84.7% 的 RV 减少。我们还评估了在现实世界图像上的性能，并展示了其优势。