Compositional zero-shot learning (CZSL) aims to recognize unseen compositions
with prior knowledge of known primitives (attribute and object). Previous works
for CZSL often suffer from grasping the contextuality between attribute and
object, as well as the discriminability of visual features, and the long-tailed
distribution of real-world compositional data. We propose a simple and scalable
framework called Composition Transformer (CoT) to address these issues. CoT
employs object and attribute experts in distinctive manners to generate
representative embeddings, using the visual network hierarchically. The object
expert extracts representative object embeddings from the final layer in a
bottom-up manner, while the attribute expert makes attribute embeddings in a
top-down manner with a proposed object-guided attention module that models
contextuality explicitly. To remedy biased prediction caused by imbalanced data
distribution, we develop a simple minority attribute augmentation (MAA) that
synthesizes virtual samples by mixing two images and oversampling minority
attribute classes. Our method achieves SoTA performance on several benchmarks,
including MIT-States, C-GQA, and VAW-CZSL. We also demonstrate the
effectiveness of CoT in improving visual discrimination and addressing the
model bias from the imbalanced data distribution. The code is available at
this https URL

通过使用 Composition Transformer（CoT）框架，我们提出了一种简单可扩展的方法来解决组合式零样本学习（CZSL）中的上下文问题、视觉特征可辨识性问题和真实世界组合数据的长尾分布问题；CoT 框架通过底层的自下而上方式提取代表性的物体嵌入，并通过顶层的自上而下方式以一种显式建模上下文性的对象引导注意力模块生成代表性的属性嵌入；我们还开发了一种简单的少数类属性增广（MAA）方法，通过混合两张图像和过采样少数类属性来合成虚拟样本，以解决不平衡的数据分布所引起的预测偏差；我们的方法在多个基准测试集上达到了最先进的性能，包括 MIT-States，C-GQA 和 VAW-CZSL；此外，我们还展示了 CoT 在提高视觉辨识能力和解决因不平衡数据分布而引起的模型偏差方面的有效性。

分层视觉基元专家的组合式零样本学习

Hierarchical Visual Primitive Experts for Compositional Zero-Shot  Learning

This paper presents an end-to-end instance segmentation framework, termed
SOIT, that Segments Objects with Instance-aware Transformers. Inspired by DETR
\cite{carion2020end}, our method views instance segmentation as a direct set
prediction problem and effectively removes the need for many hand-crafted
components like RoI cropping, one-to-many label assignment, and non-maximum
suppression (NMS). In SOIT, multiple queries are learned to directly reason a
set of object embeddings of semantic category, bounding-box location, and
pixel-wise mask in parallel under the global image context. The class and
bounding-box can be easily embedded by a fixed-length vector. The pixel-wise
mask, especially, is embedded by a group of parameters to construct a
lightweight instance-aware transformer. Afterward, a full-resolution mask is
produced by the instance-aware transformer without involving any RoI-based
operation. Overall, SOIT introduces a simple single-stage instance segmentation
framework that is both RoI- and NMS-free. Experimental results on the MS COCO
dataset demonstrate that SOIT outperforms state-of-the-art instance
segmentation approaches significantly. Moreover, the joint learning of multiple
tasks in a unified query embedding can also substantially improve the detection
performance. Code is available at https://github.com/yuxiaodongHRI/SOIT.

本文提出了一种简单的单阶段实例分割框架，SOIT，用于通过实例感知变换器对物体进行分割。

使用实例感知 Transformer 分割对象

SOIT: Segmenting Objects with Instance-Aware Transformers

Detecting spliced images is one of the emerging challenges in computer
vision. Unlike prior methods that focus on detecting low-level artifacts
generated during the manipulation process, we use an image retrieval approach
to tackle this problem. When given a spliced query image, our goal is to
retrieve the original image from a database of authentic images. To achieve
this goal, we propose representing an image by its constituent objects based on
the intuition that the finest granularity of manipulations is oftentimes at the
object-level. We introduce a framework, object embeddings for spliced image
retrieval (OE-SIR), that utilizes modern object detectors to localize object
regions. Each region is then embedded and collectively used to represent the
image. Further, we propose a student-teacher training paradigm for learning
discriminative embeddings within object regions to avoid expensive multiple
forward passes. Detailed analysis of the efficacy of different feature
embedding models is also provided in this study. Extensive experimental results
show that the OE-SIR achieves state-of-the-art performance in spliced image
retrieval.

本研究提出了一种通过对象嵌入的方式，利用近期的目标检测器来定位对象区域并在集合层面上代表图像的框架，它可以有效地解决计算机视觉领域中检测篡改图像的难题，并取得了最佳的效果。