Click-point-based interactive segmentation has received widespread attention
due to its efficiency. However, it's hard for existing algorithms to obtain
precise and robust responses after multiple clicks. In this case, the
segmentation results tend to have little change or are even worse than before.
To improve the robustness of the response, we propose a structured click intent
model based on graph neural networks, which adaptively obtains graph nodes via
the global similarity of user-clicked Transformer tokens. Then the graph nodes
will be aggregated to obtain structured interaction features. Finally, the dual
cross-attention will be used to inject structured interaction features into
vision Transformer features, thereby enhancing the control of clicks over
segmentation results. Extensive experiments demonstrated the proposed algorithm
can serve as a general structure in improving Transformer-based interactive
segmenta?tion performance. The code and data will be released at
this https URL

我们提出了一种基于图神经网络的结构点击意图模型，通过全局相似性获得用户点击的 Transformer token，然后聚合图节点以获得结构交互特征，最后使用双重交互注意力将结构交互特征注入视觉 Transformer 特征，从而增强点击对分割结果的控制，实验证明该算法能够提高基于 Transformer 的交互分割性能。

基于 Transformer 的交互式分割中的结构化点击控制

Structured Click Control in Transformer-based Interactive Segmentation

Attributed graph clustering is an unsupervised task that partitions nodes
into different groups. Self-supervised learning (SSL) shows great potential in
handling this task, and some recent studies simultaneously learn multiple SSL
tasks to further boost performance. Currently, different SSL tasks are assigned
the same set of weights for all graph nodes. However, we observe that some
graph nodes whose neighbors are in different groups require significantly
different emphases on SSL tasks. In this paper, we propose to dynamically learn
the weights of SSL tasks for different nodes and fuse the embeddings learned
from different SSL tasks to boost performance. We design an innovative graph
clustering approach, namely Dynamically Fusing Self-Supervised Learning
(DyFSS). Specifically, DyFSS fuses features extracted from diverse SSL tasks
using distinct weights derived from a gating network. To effectively learn the
gating network, we design a dual-level self-supervised strategy that
incorporates pseudo labels and the graph structure. Extensive experiments on
five datasets show that DyFSS outperforms the state-of-the-art multi-task SSL
methods by up to 8.66% on the accuracy metric. The code of DyFSS is available
at: this https URL

本文提出了一种创新的图聚类方法，名为动态融合自监督学习（DyFSS），通过动态学习不同节点的自监督学习任务的权重，并融合不同自监督学习任务的特征来提高性能。通过在五个数据集上进行广泛实验，DyFSS 在准确性指标上超过了最先进的多任务自监督学习方法，最高可达 8.66％。