3D dense captioning is a recently-proposed novel task, where point clouds
contain more geometric information than the 2D counterpart. However, it is also
more challenging due to the higher complexity and wider variety of inter-object
relations contained in point clouds. Existing methods only treat such relations
as by-products of object feature learning in graphs without specifically
encoding them, which leads to sub-optimal results. In this paper, aiming at
improving 3D dense captioning via capturing and utilizing the complex relations
in the 3D scene, we propose MORE, a Multi-Order RElation mining model, to
support generating more descriptive and comprehensive captions. Technically,
our MORE encodes object relations in a progressive manner since complex
relations can be deduced from a limited number of basic ones. We first devise a
novel Spatial Layout Graph Convolution (SLGC), which semantically encodes
several first-order relations as edges of a graph constructed over 3D object
proposals. Next, from the resulting graph, we further extract multiple triplets
which encapsulate basic first-order relations as the basic unit, and construct
several Object-centric Triplet Attention Graphs (OTAG) to infer multi-order
relations for every target object. The updated node features from OTAG are
aggregated and fed into the caption decoder to provide abundant relational
cues, so that captions including diverse relations with context objects can be
generated. Extensive experiments on the Scan2Cap dataset prove the
effectiveness of our proposed MORE and its components, and we also outperform
the current state-of-the-art method. Our code is available at
this https URL.

本文提出 MORE，一种多阶关系挖掘模型，通过逐步建立空间布局图卷积和基于对象中心的三元组注意力图来捕捉和利用三维场景中的复杂关系，以生成更全面和描述性的标题，证明了 MORE 和其组件的有效性，并超过了现有的最新方法。