We present a deep neural network to predict structural similarity between 2D
layouts by leveraging Graph Matching Networks (GMN). Our network, coined
LayoutGMN, learns the layout metric via neural graph matching, using an
attention-based GMN designed under a triplet network setting. To train our
network, we utilize weak labels obtained by pixel-wise Intersection-over-Union
(IoUs) to define the triplet loss. Importantly, LayoutGMN is built with a
structural bias which can effectively compensate for the lack of structure
awareness in IoUs. We demonstrate this on two prominent forms of layouts, viz.,
floorplans and UI designs, via retrieval experiments on large-scale datasets.
In particular, retrieval results by our network better match human judgement of
structural layout similarity compared to both IoUs and other baselines
including a state-of-the-art method based on graph neural networks and image
convolution. In addition, LayoutGMN is the first deep model to offer both
metric learning of structural layout similarity and structural matching between
layout elements.

本研究提出了 LayoutGMN，通过利用图形匹配网络（GMN）来预测 2D 布局之间的结构相似性，该网络学习布局度量并使用基于注意力的 GMN 进行三元组网络设置下的神经图匹配。通过在大规模数据集上进行检索实验，特别是对楼层平面图和 UI 设计进行检索实验，我们证明了 LayoutGMN 相对于 IoUs 和其他基线的检索结果更符合人类对结构布局相似性的判断。此外，LayoutGMN 是第一个能够提供结构布局相似性度量和布局元素之间的结构匹配的深度模型。