Semantic segmentation and semantic edge detection can be seen as two dual
problems with close relationships in computer vision. Despite the fast
evolution of learning-based 3D semantic segmentation methods, little attention
has been drawn to the learning of 3D semantic edge detectors, even less to a
joint learning method for the two tasks. In this paper, we tackle the 3D
semantic edge detection task for the first time and present a new two-stream
fully-convolutional network that jointly performs the two tasks. In particular,
we design a joint refinement module that explicitly wires region information
and edge information to improve the performances of both tasks. Further, we
propose a novel loss function that encourages the network to produce semantic
segmentation results with better boundaries. Extensive evaluations on S3DIS and
ScanNet datasets show that our method achieves on par or better performance
than the state-of-the-art methods for semantic segmentation and outperforms the
baseline methods for semantic edge detection. Code release:
this https URL

本研究通过设计一个新的两流全卷积网络，提出了一个联合学习方法来同时处理 3D 语义分割和 3D 语义边缘检测问题，特别是设计了一个联合调优模块来明确地连接区域信息和边缘信息，通过提出的新型损失函数，实现了更好的语义分割结果的同时改善边缘信息，结果显示该方法在 S3DIS 和 ScanNet 数据集上获得了与最先进方法相当或更好的性能，优于语义边缘检测的基准方法。

JSENet: 用于 3D 点云的联合语义分割和边缘检测网络

JSENet: Joint Semantic Segmentation and Edge Detection Network for 3D  Point Clouds

Features from multiple scales can greatly benefit the semantic edge detection
task if they are well fused. However, the prevalent semantic edge detection
methods apply a fixed weight fusion strategy where images with different
semantics are forced to share the same weights, resulting in universal fusion
weights for all images and locations regardless of their different semantics or
local context. In this work, we propose a novel dynamic feature fusion strategy
that assigns different fusion weights for different input images and locations
adaptively. This is achieved by a proposed weight learner to infer proper
fusion weights over multi-level features for each location of the feature map,
conditioned on the specific input. In this way, the heterogeneity in
contributions made by different locations of feature maps and input images can
be better considered and thus help produce more accurate and sharper edge
predictions. We show that our model with the novel dynamic feature fusion is
superior to fixed weight fusion and also the na\"ive location-invariant weight
fusion methods, via comprehensive experiments on benchmarks Cityscapes and SBD.
In particular, our method outperforms all existing well established methods and
achieves new state-of-the-art.

本文提出了一种新颖的动态特征融合策略，通过权重学习器对不同输入图像和位置适应地推断各级特征的融合权重，以更好地考虑由 Feature Map 的不同位置和输入图像产生的贡献，从而实现更准确、更清晰的边缘预测，实验证明该方法在语义边缘检测方面表现优异，达到了新的最高水平。

语义边缘检测的动态特征融合

Dynamic Feature Fusion for Semantic Edge Detection

Semantic edge detection (SED), which aims at jointly extracting edges as well
as their category information, has far-reaching applications in domains such as
semantic segmentation, object proposal generation, and object recognition. SED
naturally requires achieving two distinct supervision targets: locating fine
detailed edges and identifying high-level semantics. Our motivation comes from
the hypothesis that such distinct targets prevent state-of-the-art SED methods
from effectively using deep supervision to improve results. To this end, we
propose a novel fully convolutional neural network using diverse deep
supervision (DDS) within a multi-task framework where bottom layers aim at
generating category-agnostic edges, while top layers are responsible for the
detection of category-aware semantic edges. To overcome the hypothesized
supervision challenge, a novel information converter unit is introduced, whose
effectiveness has been extensively evaluated on SBD and Cityscapes datasets.

本篇论文提出了一种用于语义边缘检测的卷积神经网络，使用多种深度监督的方式对分类不可知和分类可知的边缘进行单独识别以实现更好的结果，并引入信息转换单元以解决监督挑战。