Anomaly detection (AD) aims to identify defective images and localize their
defects (if any). Ideally, AD models should be able to detect defects over many
image classes; without relying on hard-coded class names that can be
uninformative or inconsistent across datasets; learn without anomaly
supervision; and be robust to the long-tailed distributions of real-world
applications. To address these challenges, we formulate the problem of
long-tailed AD by introducing several datasets with different levels of class
imbalance and metrics for performance evaluation. We then propose a novel
method, LTAD, to detect defects from multiple and long-tailed classes, without
relying on dataset class names. LTAD combines AD by reconstruction and semantic
AD modules. AD by reconstruction is implemented with a transformer-based
reconstruction module. Semantic AD is implemented with a binary classifier,
which relies on learned pseudo class names and a pretrained foundation model.
These modules are learned over two phases. Phase 1 learns the pseudo-class
names and a variational autoencoder (VAE) for feature synthesis that augments
the training data to combat long-tails. Phase 2 then learns the parameters of
the reconstruction and classification modules of LTAD. Extensive experiments
using the proposed long-tailed datasets show that LTAD substantially
outperforms the state-of-the-art methods for most forms of dataset imbalance.
The long-tailed dataset split is available at
this https URL .

用于检测多个类别和长尾类别的缺陷的一种新方法，将异常检测与重构模块以及语义异常检测模块相结合，利用学习到的伪类名和预训练的基础模型进行二进制分类，通过两个阶段的学习，可以有效地解决真实应用中长尾数据分布的问题，并在大多数形式的数据集不平衡中显著优于其他最先进的方法。

可学习的类别名称的长尾异常检测

Long-Tailed Anomaly Detection with Learnable Class Names

Syntax is usually studied in the realm of linguistics and refers to the
arrangement of words in a sentence. Similarly, an image can be considered as a
visual 'sentence', with the semantic parts of the image acting as 'words'.
While visual syntactic understanding occurs naturally to humans, it is
interesting to explore whether deep neural networks (DNNs) are equipped with
such reasoning. To that end, we alter the syntax of natural images (e.g.
swapping the eye and nose of a face), referred to as 'incorrect' images, to
investigate the sensitivity of DNNs to such syntactic anomaly. Through our
experiments, we discover an intriguing property of DNNs where we observe that
state-of-the-art convolutional neural networks, as well as vision transformers,
fail to discriminate between syntactically correct and incorrect images when
trained on only correct ones. To counter this issue and enable visual syntactic
understanding with DNNs, we propose a three-stage framework- (i) the 'words'
(or the sub-features) in the image are detected, (ii) the detected words are
sequentially masked and reconstructed using an autoencoder, (iii) the original
and reconstructed parts are compared at each location to determine syntactic
correctness. The reconstruction module is trained with BERT-like masked
autoencoding for images, with the motivation to leverage language model
inspired training to better capture the syntax. Note, our proposed approach is
unsupervised in the sense that the incorrect images are only used during
testing and the correct versus incorrect labels are never used for training. We
perform experiments on CelebA, and AFHQ datasets and obtain classification
accuracy of 92.10%, and 90.89%, respectively. Notably, the approach generalizes
well to ImageNet samples which share common classes with CelebA and AFHQ
without explicitly training on them.

通过改变自然图像的句法（例如交换一个脸部的眼睛和鼻子）来调查深度神经网络对此类句法异常的敏感性，并提出了一个三阶段框架来实现深度神经网络的视觉句法理解。通过使用类似于 BERT 的图像掩蔽自编码器进行训练，我们在 CelebA 和 AFHQ 数据集上进行实验证明了方法的广义性能。

朝向视觉语法理解

Towards Visual Syntactical Understanding

We introduce a novel bilateral reference framework (***BiRefNet***) for
high-resolution dichotomous image segmentation (DIS). It comprises two
essential components: the localization module (LM) and the reconstruction
module (RM) with our proposed bilateral reference (BiRef). The LM aids in
object localization using global semantic information. Within the RM, we
utilize BiRef for the reconstruction process, where hierarchical patches of
images provide the source reference and gradient maps serve as the target
reference. These components collaborate to generate the final predicted maps.
We also introduce auxiliary gradient supervision to enhance focus on regions
with finer details. Furthermore, we outline practical training strategies
tailored for DIS to improve map quality and training process. To validate the
general applicability of our approach, we conduct extensive experiments on four
tasks to evince that *BiRefNet* exhibits remarkable performance, outperforming
task-specific cutting-edge methods across all benchmarks.

我们引入了一种新的双边参考框架（BiRefNet），用于高分辨率的二分图像分割（DIS），通过全局语义信息的方式来进行对象定位，利用双边参考进行图像重建，并引入辅助梯度监督来提高对细节区域的关注，这种方法在多个任务中展现非凡的性能，超过了各种基准上的最新方法。