Interactive segmentation methods rely on user inputs to iteratively update
the selection mask. A click specifying the object of interest is arguably the
most simple and intuitive interaction type, and thereby the most common choice
for interactive segmentation. However, user clicking patterns in the
interactive segmentation context remain unexplored. Accordingly, interactive
segmentation evaluation strategies rely more on intuition and common sense
rather than empirical studies (e.g., assuming that users tend to click in the
center of the area with the largest error). In this work, we conduct a real
user study to investigate real user clicking patterns. This study reveals that
the intuitive assumption made in the common evaluation strategy may not hold.
As a result, interactive segmentation models may show high scores in the
standard benchmarks, but it does not imply that they would perform well in a
real world scenario. To assess the applicability of interactive segmentation
methods, we propose a novel evaluation strategy providing a more comprehensive
analysis of a model's performance. To this end, we propose a methodology for
finding extreme user inputs by a direct optimization in a white-box adversarial
attack on the interactive segmentation model. Based on the performance with
such adversarial user inputs, we assess the robustness of interactive
segmentation models w.r.t click positions. Besides, we introduce a novel
benchmark for measuring the robustness of interactive segmentation, and report
the results of an extensive evaluation of dozens of models.

通过对交互式分割模型的实际用户研究，发现常规评估策略的直观假设可能无法成立，因此建议采用直接优化的白盒对抗攻击方法来评估交互式分割模型的鲁棒性，并引入了一种新的鲁棒性度量标准和多个模型的广泛评估结果。