In this paper we address three different computer vision tasks using a single
basic architecture: depth prediction, surface normal estimation, and semantic
labeling. We use a multiscale convolutional network that is able to adapt
easily to each task using only small modifications, regressing from the input
image to the output map directly. Our method progressively refines predictions
using a sequence of scales, and captures many image details without any
superpixels or low-level segmentation. We achieve state-of-the-art performance
on benchmarks for all three tasks.

本文提出了一个利用多尺度卷积神经网络实现计算机视觉中深度预测、表面法线估计和语义标签等任务的方法，并在各项任务的基准测试中实现了最优表现。

使用共同的多尺度卷积结构预测深度，表面法线和语义标签

Predicting Depth, Surface Normals and Semantic Labels with a Common  Multi-Scale Convolutional Architecture

This work addresses multi-class segmentation of indoor scenes with RGB-D
inputs. While this area of research has gained much attention recently, most
works still rely on hand-crafted features. In contrast, we apply a multiscale
convolutional network to learn features directly from the images and the depth
information. We obtain state-of-the-art on the NYU-v2 depth dataset with an
accuracy of 64.5%. We illustrate the labeling of indoor scenes in videos
sequences that could be processed in real-time using appropriate hardware such
as an FPGA.

采用多尺度卷积神经网络直接从图像和深度信息中学习特征实现室内场景的多类别分割，并在 NYU-v2 深度数据集上取得了 64.5% 的准确率，充分说明了适当的硬件（如 FPGA）可以实现实时处理。

室内语义分割使用深度信息

Indoor Semantic Segmentation using depth information

Scene parsing, or semantic segmentation, consists in labeling each pixel in
an image with the category of the object it belongs to. It is a challenging
task that involves the simultaneous detection, segmentation and recognition of
all the objects in the image.
The scene parsing method proposed here starts by computing a tree of segments
from a graph of pixel dissimilarities. Simultaneously, a set of dense feature
vectors is computed which encodes regions of multiple sizes centered on each
pixel. The feature extractor is a multiscale convolutional network trained from
raw pixels. The feature vectors associated with the segments covered by each
node in the tree are aggregated and fed to a classifier which produces an
estimate of the distribution of object categories contained in the segment. A
subset of tree nodes that cover the image are then selected so as to maximize
the average "purity" of the class distributions, hence maximizing the overall
likelihood that each segment will contain a single object. The convolutional
network feature extractor is trained end-to-end from raw pixels, alleviating
the need for engineered features. After training, the system is parameter free.
The system yields record accuracies on the Stanford Background Dataset (8
classes), the Sift Flow Dataset (33 classes) and the Barcelona Dataset (170
classes) while being an order of magnitude faster than competing approaches,
producing a 320 \times 240 image labeling in less than 1 second.

将场景解析或语义分割应用于图像，使用从原始像素训练的多尺度卷积网络作为特征提取器，将图像分割成多个区域，并对每个区域分类，从而实现对象检测和图像标注。