Recent progress on deep learning has made it possible to automatically
transform the screenshot of Graphic User Interface (GUI) into code by using the
encoder-decoder framework. While the commonly adopted image encoder (e.g., CNN
network), might be capable of extracting image features to the desired level,
interpreting these abstract image features into hundreds of tokens of code puts
a particular challenge on the decoding power of the RNN-based code generator.
Considering the code used for describing GUI is usually hierarchically
structured, we propose a new attention-based hierarchical code generation
model, which can describe GUI images in a finer level of details, while also
being able to generate hierarchically structured code in consistency with the
hierarchical layout of the graphic elements in the GUI. Our model follows the
encoder-decoder framework, all the components of which can be trained jointly
in an end-to-end manner. The experimental results show that our method
outperforms other current state-of-the-art methods on both a publicly available
GUI-code dataset as well as a dataset established by our own.

该论文提出了一种基于注意力机制的分层代码生成模型，可更细致地描述图形用户界面 (GUI) 中的图片，并生成与 GUI 中图形元素的分层布局一致的代码，通过实验证明该模型优于当下其他最先进的方法。