Extracting semantic representations from mobile user interfaces (UI) and
using the representations for designers' decision-making processes have shown
the potential to be effective computational design support tools. Current
approaches rely on machine learning models trained on small-sized mobile UI
datasets to extract semantic vectors and use screenshot-to-screenshot
comparison to retrieve similar-looking UIs given query screenshots. However,
the usability of these methods is limited because they are often not
open-sourced and have complex training pipelines for practitioners to follow,
and are unable to perform screenshot set-to-set (i.e., app-to-app) retrieval.
To this end, we (1) employ visual models trained with large web-scale images
and test whether they could extract a UI representation in a zero-shot way and
outperform existing specialized models, and (2) use mathematically founded
methods to enable app-to-app retrieval and design consistency analysis. Our
experiments show that our methods not only improve upon previous retrieval
models but also enable multiple new applications.

从移动用户界面 (UI) 中提取语义表示并将这些表示用于设计师的决策过程已显示出成为有效的计算设计支持工具的潜力。我们采用大规模网络图像训练的视觉模型，以零 - shot 方式提取 UI 表示并超越现有专门模型，并使用数学基础的方法实现应用程序之间的检索和设计一致性分析。我们的实验表明，我们的方法不仅改进了以前的检索模型，还实现了多个新的应用。