Omnidirectional vision is becoming increasingly relevant as more efficient
$360^o$ image acquisition is now possible. However, the lack of annotated
$360^o$ datasets has hindered the application of deep learning techniques on
spherical content. This is further exaggerated on tasks where ground truth
acquisition is difficult, such as monocular surface estimation. While recent
research approaches on the 2D domain overcome this challenge by relying on
generating normals from depth cues using RGB-D sensors, this is very difficult
to apply on the spherical domain. In this work, we address the unavailability
of sufficient $360^o$ ground truth normal data, by leveraging existing 3D
datasets and remodelling them via rendering. We present a dataset of $360^o$
images of indoor spaces with their corresponding ground truth surface normal,
and train a deep convolutional neural network (CNN) on the task of monocular
360 surface estimation. We achieve this by minimizing a novel angular loss
function defined on the hyper-sphere using simple quaternion algebra. We put an
effort to appropriately compare with other state of the art methods trained on
planar datasets and finally, present the practical applicability of our trained
model on a spherical image re-lighting task using completely unseen data by
qualitatively showing the promising generalization ability of our dataset and
model. The dataset is available at:
vcl3d.github.io/HyperSphereSurfaceRegression.

本文通过利用现有的 3D 数据集和重新渲染的方式解决 $360^o$ 图像的表面法向数据不足的问题，并通过深度卷积神经网络来实现全景图像的表面估计。

超球损失的 $360^o$ 表面回归

$360^o$ Surface Regression with a Hyper-Sphere Loss

Convolutional Neural Networks (CNNs) have become the method of choice for
learning problems involving 2D planar images. However, a number of problems of
recent interest have created a demand for models that can analyze spherical
images. Examples include omnidirectional vision for drones, robots, and
autonomous cars, molecular regression problems, and global weather and climate
modelling. A naive application of convolutional networks to a planar projection
of the spherical signal is destined to fail, because the space-varying
distortions introduced by such a projection will make translational weight
sharing ineffective.
In this paper we introduce the building blocks for constructing spherical
CNNs. We propose a definition for the spherical cross-correlation that is both
expressive and rotation-equivariant. The spherical correlation satisfies a
generalized Fourier theorem, which allows us to compute it efficiently using a
generalized (non-commutative) Fast Fourier Transform (FFT) algorithm. We
demonstrate the computational efficiency, numerical accuracy, and effectiveness
of spherical CNNs applied to 3D model recognition and atomization energy
regression.

该研究介绍了构建球形 CNN 的基本模块，该模型可用于解决分子回归和全球气候建模等领域的问题，并通过实验证明了其计算效率、数值精度和有效性。