We present a task and benchmark dataset for person-centric visual grounding,
the problem of linking between people named in a caption and people pictured in
an image. In contrast to prior work in visual grounding, which is predominantly
object-based, our new task masks out the names of people in captions in order
to encourage methods trained on such image-caption pairs to focus on contextual
cues (such as rich interactions between multiple people), rather than learning
associations between names and appearances. To facilitate this task, we
introduce a new dataset, Who's Waldo, mined automatically from image-caption
data on Wikimedia Commons. We propose a Transformer-based method that
outperforms several strong baselines on this task, and are releasing our data
to the research community to spur work on contextual models that consider both
vision and language.

我们提出了一个任务和基准数据集，用于个体中心的视觉定位，这个问题是将标题中命名的人物与图像中出现的人物联系起来。为了鼓励方法关注上下文线索（如多个人之间的丰富互动），而不是学习名称和外观之间的关联，我们的新任务遮盖了标题中的人名。为了促进这个任务，我们介绍了一个新的数据集，谁是沃尔多，在维基共享资源的图像标题数据上自动挖掘。我们提出了一种基于 Transformer 的方法，在这个任务上优于几种强基线，并向研究社区发布我们的数据，以激发关注视觉和语言的上下文模型的工作。

谁是沃尔多？链接文本和图像中的人物

Who's Waldo? Linking People Across Text and Images

Due to the fact that fully supervised semantic segmentation methods require
sufficient fully-labeled data to work well and can not generalize to unseen
classes, few-shot segmentation has attracted lots of research attention.
Previous arts extract features from support and query images, which are
processed jointly before making predictions on query images. The whole process
is based on convolutional neural networks (CNN), leading to the problem that
only local information is used. In this paper, we propose a TRansformer-based
Few-shot Semantic segmentation method (TRFS). Specifically, our model consists
of two modules: Global Enhancement Module (GEM) and Local Enhancement Module
(LEM). GEM adopts transformer blocks to exploit global information, while LEM
utilizes conventional convolutions to exploit local information, across query
and support features. Both GEM and LEM are complementary, helping to learn
better feature representations for segmenting query images. Extensive
experiments on PASCAL-5i and COCO datasets show that our approach achieves new
state-of-the-art performance, demonstrating its effectiveness.

本文提出了一种基于 TRansformer 网络的 Few-shot 语义分割方法（TRFS），它采用全局增强模块（GEM）和局部增强模块（LEM）相结合的方法，可以在 PASCAL-5i 和 COCO 数据集上取得新的最优表现。

利用 Transformer 提升 Few-shot 语义分割

Boosting Few-shot Semantic Segmentation with Transformers

Temporal action detection (TAD) aims to determine the semantic label and the
temporal interval of every action instance in an untrimmed video. It is a
fundamental and challenging task in video understanding. Previous methods
tackle this task with complicated pipelines. They often need to train multiple
networks and involve hand-designed operations, such as non-maximal suppression
and anchor generation, which limit the flexibility and prevent end-to-end
learning. In this paper, we propose an end-to-end Transformer-based method for
TAD, termed TadTR. Given a small set of learnable embeddings called action
queries, TadTR adaptively extracts temporal context information from the video
for each query and directly predicts action instances with the context. To
adapt Transformer to TAD, we propose three improvements to enhance its locality
awareness. The core is a temporal deformable attention module that selectively
attends to a sparse set of key snippets in a video. A segment refinement
mechanism and an actionness regression head are designed to refine the
boundaries and confidence of the predicted instances, respectively. With such a
simple pipeline, TadTR requires lower computation cost than previous detectors,
while preserving remarkable performance. As a self-contained detector, it
achieves state-of-the-art performance on THUMOS14 (56.7% mAP) and HACS Segments
(32.09% mAP). Combined with an extra action classifier, it obtains 36.75% mAP
on ActivityNet-1.3. Code is available at this https URL

本研究提出了一种基于 Transformer 的时态动作检测方法，通过少量学习嵌入（称为动作查询）从视频中自适应地提取时间上下文信息，并直接预测动作实例的时间间隔和语义标签，通过三种改进提高本地化意识，实现了端到端的学习，并取得了与现有方法相比的卓越性能。