Convolutional Neural Networks (CNN) are successfully used for various visual
perception tasks including bounding box object detection, semantic
segmentation, optical flow, depth estimation and visual SLAM. Generally these
tasks are independently explored and modeled. In this paper, we present a joint
multi-task network design for learning object detection and semantic
segmentation simultaneously. The main motivation is to achieve real-time
performance on a low power embedded SOC by sharing of encoder for both the
tasks. We construct an efficient architecture using a small ResNet10 like
encoder which is shared for both decoders. Object detection uses YOLO v2 like
decoder and semantic segmentation uses FCN8 like decoder. We evaluate the
proposed network in two public datasets (KITTI, Cityscapes) and in our private
fisheye camera dataset, and demonstrate that joint network provides the same
accuracy as that of separate networks. We further optimize the network to
achieve 30 fps for 1280x384 resolution image.

本论文提出了一种联合多任务网络设计，用于同时学习目标检测和语义分割，以实现低功耗嵌入式 SOC 上的实时性能，并在两个公共数据集（KITTI，Cityscapes）和私人鱼眼相机数据集中评估提出的网络。