Current high-performance semantic segmentation models are purely data-driven
sub-symbolic approaches and blind to the structured nature of the visual world.
This is in stark contrast to human cognition which abstracts visual perceptions
at multiple levels and conducts symbolic reasoning with such structured
abstraction. To fill these fundamental gaps, we devise LOGICSEG, a holistic
visual semantic parser that integrates neural inductive learning and logic
reasoning with both rich data and symbolic knowledge. In particular, the
semantic concepts of interest are structured as a hierarchy, from which a set
of constraints are derived for describing the symbolic relations and formalized
as first-order logic rules. After fuzzy logic-based continuous relaxation,
logical formulae are grounded onto data and neural computational graphs, hence
enabling logic-induced network training. During inference, logical constraints
are packaged into an iterative process and injected into the network in a form
of several matrix multiplications, so as to achieve hierarchy-coherent
prediction with logic reasoning. These designs together make LOGICSEG a general
and compact neural-logic machine that is readily integrated into existing
segmentation models. Extensive experiments over four datasets with various
segmentation models and backbones verify the effectiveness and generality of
LOGICSEG. We believe this study opens a new avenue for visual semantic parsing.

LOGICSEG 是一个综合的视觉语义解析器，它集成了神经归纳学习和逻辑推理，通过丰富的数据和符号知识，将视觉概念结构化为一个层级结构，进行层次一致的预测和逻辑推理，并通过多个数据和神经计算图来进行逻辑感知网络的训练，从而实现与现有分割模型的集成。本研究证实了 LOGICSEG 的有效性和普适性，为视觉语义解析开辟了新的途径。