Depth estimation is a crucial step for 3D reconstruction with panorama images
in recent years. Panorama images maintain the complete spatial information but
introduce distortion with equirectangular projection. In this paper, we propose
an ACDNet based on the adaptively combined dilated convolution to predict the
dense depth map for a monocular panoramic image. Specifically, we combine the
convolution kernels with different dilations to extend the receptive field in
the equirectangular projection. Meanwhile, we introduce an adaptive
channel-wise fusion module to summarize the feature maps and get diverse
attention areas in the receptive field along the channels. Due to the
utilization of channel-wise attention in constructing the adaptive channel-wise
fusion module, the network can capture and leverage the cross-channel
contextual information efficiently. Finally, we conduct depth estimation
experiments on three datasets (both virtual and real-world) and the
experimental results demonstrate that our proposed ACDNet substantially
outperforms the current state-of-the-art (SOTA) methods. Our codes and model
parameters are accessed in this https URL

本文提出了一种基于自适应膨胀卷积的 ACDNet，用于预测单目全景图像的密集深度图，实验结果表明 ACDNet 明显优于当前最先进的技术。

ACDNet：自适应组合扩张卷积用于单目全景深度估计

ACDNet: Adaptively Combined Dilated Convolution for Monocular Panorama  Depth Estimation

Panorama images have a much larger field-of-view thus naturally encode
enriched scene context information compared to standard perspective images,
which however is not well exploited in the previous scene understanding
methods. In this paper, we propose a novel method for panoramic 3D scene
understanding which recovers the 3D room layout and the shape, pose, position,
and semantic category for each object from a single full-view panorama image.
In order to fully utilize the rich context information, we design a novel graph
neural network based context model to predict the relationship among objects
and room layout, and a differentiable relationship-based optimization module to
optimize object arrangement with well-designed objective functions on-the-fly.
Realizing the existing data are either with incomplete ground truth or
overly-simplified scene, we present a new synthetic dataset with good diversity
in room layout and furniture placement, and realistic image quality for total
panoramic 3D scene understanding. Experiments demonstrate that our method
outperforms existing methods on panoramic scene understanding in terms of both
geometry accuracy and object arrangement. Code is available at
this https URL

本研究提出了一种基于全景图像的新颖的三维场景理解方法，通过设计基于图神经网络的上下文模型来预测物体之间的关系和房间布局，以及可微分的基于关系的优化模块来优化物体排列，实现对全景三维场景的理解， 并制备了具有多样化房间布局和家具摆放的真实感合成数据集，在几何精度和目标布置方面优于现有方法。

DeepPanoContext: 全景三维场景理解，基于整体场景上下文图和基于关系的优化

DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene  Context Graph and Relation-based Optimization

Convolutional neural networks (CNNs) have been widely used in various vision
tasks, e.g. image classification, semantic segmentation, etc. Unfortunately,
standard 2D CNNs are not well suited for spherical signals such as panorama
images or spherical projections, as the sphere is an unstructured grid. In this
paper, we present Spherical Transformer which can transform spherical signals
into vectors that can be directly processed by standard CNNs such that many
well-designed CNNs architectures can be reused across tasks and datasets by
pretraining. To this end, the proposed method first uses local structured
sampling methods such as HEALPix to construct a transformer grid by using the
information of spherical points and its adjacent points, and then transforms
the spherical signals to the vectors through the grid. By building the
Spherical Transformer module, we can use multiple CNN architectures directly.
We evaluate our approach on the tasks of spherical MNIST recognition, 3D object
classification and omnidirectional image semantic segmentation. For 3D object
classification, we further propose a rendering-based projection method to
improve the performance and a rotational-equivariant model to improve the
anti-rotation ability. Experimental results on three tasks show that our
approach achieves superior performance over state-of-the-art methods.

本文提出基于 Spherical Transformer 的方法，将球面信号转换为能够被标准 CNNs 直接处理的向量，从而使许多经过精心设计的 CNNs 架构可以通过预训练在不同任务和数据集中重复使用，该方法在球形 MNIST 识别，3D 物体分类和全向图像语义分割任务上具有优异性能。