Present-day graphical user interfaces (GUIs) exhibit diverse arrangements of
text, graphics, and interactive elements such as buttons and menus, but
representations of GUIs have not kept up. They do not encapsulate both semantic
and visuo-spatial relationships among elements. To seize machine learning's
potential for GUIs more efficiently, Graph4GUI exploits graph neural networks
to capture individual elements' properties and their semantic-visuo-spatial
constraints in a layout. The learned representation demonstrated its
effectiveness in multiple tasks, especially generating designs in a challenging
GUI autocompletion task, which involved predicting the positions of remaining
unplaced elements in a partially completed GUI. The new model's suggestions
showed alignment and visual appeal superior to the baseline method and received
higher subjective ratings for preference. Furthermore, we demonstrate the
practical benefits and efficiency advantages designers perceive when utilizing
our model as an autocompletion plug-in.

现代的图形用户界面（GUI）展示着不同的文本、图形和交互元素的排列，但 GUI 的表达方式却没有跟上。为了更高效地利用机器学习在 GUI 中的潜力，Graph4GUI 利用图神经网络捕捉了各个元素的属性以及它们之间的语义 - 视觉 - 空间约束关系。通过多项任务的效果验证，学习到的表示尤其在具有挑战性的 GUI 自动补全任务中表现出了其有效性，该任务涉及在部分完成的 GUI 中预测剩余未放置元素的位置。与基准方法相比，新模型的建议显示出更好的对齐和视觉吸引力，并且在主观评分上获得了更高的偏好度。此外，我们还展示了设计师在使用我们的模型作为自动补全插件时所感知到的实际效益和效率优势。