Different from the context-independent (CI) concepts such as human, car, and
airplane, context-dependent (CD) concepts require higher visual understanding
ability, such as camouflaged object and medical lesion. Despite the rapid
advance of many CD understanding tasks in respective branches, the isolated
evolution leads to their limited cross-domain generalisation and repetitive
technique innovation. Since there is a strong coupling relationship between
foreground and background context in CD tasks, existing methods require to
train separate models in their focused domains. This restricts their real-world
CD concept understanding towards artificial general intelligence (AGI). We
propose a unified model with a single set of parameters, Spider, which only
needs to be trained once. With the help of the proposed concept filter driven
by the image-mask group prompt, Spider is able to understand and distinguish
diverse strong context-dependent concepts to accurately capture the Prompter's
intention. Without bells and whistles, Spider significantly outperforms the
state-of-the-art specialized models in 8 different context-dependent
segmentation tasks, including 4 natural scenes (salient, camouflaged, and
transparent objects and shadow) and 4 medical lesions (COVID-19, polyp, breast,
and skin lesion with color colonoscopy, CT, ultrasound, and dermoscopy
modalities). Besides, Spider shows obvious advantages in continuous learning.
It can easily complete the training of new tasks by fine-tuning parameters less
than 1\% and bring a tolerable performance degradation of less than 5\% for all
old tasks. The source code will be publicly available at
\href{this https URL}{Spider-UniCDSeg}.

提出了一个单一模型 Spider，用于理解和区分不同的强依赖上下文概念，以准确捕捉指示器的意图，并在 8 个不同的依赖上下文分割任务以及连续学习方面显现明显的优势。

蜘蛛：一种用于上下文相关概念理解的统一框架

Spider: A Unified Framework for Context-dependent Concept Understanding

Dense retrievers for open-domain question answering (ODQA) have been shown to
achieve impressive performance by training on large datasets of
question-passage pairs. In this work we ask whether this dependence on labeled
data can be reduced via unsupervised pretraining that is geared towards ODQA.
We show this is in fact possible, via a novel pretraining scheme designed for
retrieval. Our "recurring span retrieval" approach uses recurring spans across
passages in a document to create pseudo examples for contrastive learning. Our
pretraining scheme directly controls for term overlap across pseudo queries and
relevant passages, thus allowing to model both lexical and semantic relations
between them. The resulting model, named Spider, performs surprisingly well
without any labeled training examples on a wide range of ODQA datasets.
Specifically, it significantly outperforms all other pretrained baselines in a
zero-shot setting, and is competitive with BM25, a strong sparse baseline.
Moreover, a hybrid retriever over Spider and BM25 improves over both, and is
often competitive with DPR models, which are trained on tens of thousands of
examples. Last, notable gains are observed when using Spider as an
initialization for supervised training.

本文介绍了一种基于无监督预训练的 ODQA 方法，通过 recurrent span retrieval 从文档中创建伪例子进行对比学习，控制 pseudo 查询和相关段落之间的术语重叠，从而允许模拟它们之间的词汇和语义关系，得到命名为 “Spider” 的模型，具有出色的性能，且不需要任何有标签的训练数据。