We use a deep learning based approach to predict whether a selected element
in a mobile UI screenshot will be perceived by users as tappable, based on
pixels only instead of view hierarchies required by previous work. To help
designers better understand model predictions and to provide more actionable
design feedback than predictions alone, we additionally use ML interpretability
techniques to help explain the output of our model. We use XRAI to highlight
areas in the input screenshot that most strongly influence the tappability
prediction for the selected region, and use k-Nearest Neighbors to present the
most similar mobile UIs from the dataset with opposing influences on
tappability perception.

本文运用基于深度学习的方法，能够仅基于像素预测移动 UI 截屏中被用户感知为可点击的元素，并运用 ML 可解释性技术解释模型输出结果，其中采用 XRAI 技术突出显示对所选区域的可点击性预测影响最强烈的截屏区域，采用 k - 最近邻方法表示数据集中对可点击性感知产生相反影响的最相似移动 UI。

利用视觉建模和显著性分析预测和解释移动 UI 的可点击性

Predicting and Explaining Mobile UI Tappability with Vision Modeling and Saliency Analysis

Saliency methods can aid understanding of deep neural networks. Recent years
have witnessed many improvements to saliency methods, as well as new ways for
evaluating them. In this paper, we 1) present a novel region-based attribution
method, XRAI, that builds upon integrated gradients (Sundararajan et al. 2017),
2) introduce evaluation methods for empirically assessing the quality of
image-based saliency maps (Performance Information Curves (PICs)), and 3)
contribute an axiom-based sanity check for attribution methods. Through
empirical experiments and example results, we show that XRAI produces better
results than other saliency methods for common models and the ImageNet dataset.

本文提出了一种基于区域的新型归因方法 XRAI，介绍了评估方法 PICs，并且通过实验展示在常见模型和 ImageNet 数据集上，XRAI 比其他显著性方法产生更好的结果。