User interface (UI) design is a difficult yet important task for ensuring the
usability, accessibility, and aesthetic qualities of applications. In our
paper, we develop a machine-learned model, UIClip, for assessing the design
quality and visual relevance of a UI given its screenshot and natural language
description. To train UIClip, we used a combination of automated crawling,
synthetic augmentation, and human ratings to construct a large-scale dataset of
UIs, collated by description and ranked by design quality. Through training on
the dataset, UIClip implicitly learns properties of good and bad designs by i)
assigning a numerical score that represents a UI design's relevance and quality
and ii) providing design suggestions. In an evaluation that compared the
outputs of UIClip and other baselines to UIs rated by 12 human designers, we
found that UIClip achieved the highest agreement with ground-truth rankings.
Finally, we present three example applications that demonstrate how UIClip can
facilitate downstream applications that rely on instantaneous assessment of UI
design quality: i) UI code generation, ii) UI design tips generation, and iii)
quality-aware UI example search.

通过机器学习模型 UIClip，本研究开发了一种用于评估 UI 设计质量和视觉相关性的方法，通过截图和自然语言描述来对 UI 进行评估，实验证明 UIClip 在与其他基准方法和人工设计师评分进行对比时取得了最高一致性，从而为 UI 设计质量的直接评估提供了便利。

UIClip：一个基于数据的用户界面设计评估模型

UIClip: A Data-driven Model for Assessing User Interface Design

Model-based optimization (MBO) is increasingly applied to design problems in
science and engineering. A common scenario involves using a fixed training set
to train models, with the goal of designing new samples that outperform those
present in the training data. A major challenge in this setting is distribution
shift, where the distributions of training and design samples are different.
While some shift is expected, as the goal is to create better designs, this
change can negatively affect model accuracy and subsequently, design quality.
Despite the widespread nature of this problem, addressing it demands deep
domain knowledge and artful application. To tackle this issue, we propose a
straightforward method for design practitioners that detects distribution
shifts. This method trains a binary classifier using knowledge of the unlabeled
design distribution to separate the training data from the design data. The
classifier's logit scores are then used as a proxy measure of distribution
shift. We validate our method in a real-world application by running offline
MBO and evaluate the effect of distribution shift on design quality. We find
that the intensity of the shift in the design distribution varies based on the
number of steps taken by the optimization algorithm, and our simple approach
can identify these shifts. This enables users to constrain their search to
regions where the model's predictions are reliable, thereby increasing the
quality of designs.

通过训练二元分类器，我们提出了一种简单的方法来检测分布的变化，从而解决了在模型设计过程中的分布偏移问题，并通过验证其在实际应用中的有效性，证实了此方法可以提高设计的质量。