While originally designed for image generation, diffusion models have
recently shown to provide excellent pretrained feature representations for
semantic segmentation. Intrigued by this result, we set out to explore how well
diffusion-pretrained representations generalize to new domains, a crucial
ability for any representation. We find that diffusion-pretraining achieves
extraordinary domain generalization results for semantic segmentation,
outperforming both supervised and self-supervised backbone networks. Motivated
by this, we investigate how to utilize the model's unique ability of taking an
input prompt, in order to further enhance its cross-domain performance. We
introduce a scene prompt and a prompt randomization strategy to help further
disentangle the domain-invariant information when training the segmentation
head. Moreover, we propose a simple but highly effective approach for test-time
domain adaptation, based on learning a scene prompt on the target domain in an
unsupervised manner. Extensive experiments conducted on four synthetic-to-real
and clear-to-adverse weather benchmarks demonstrate the effectiveness of our
approaches. Without resorting to any complex techniques, such as image
translation, augmentation, or rare-class sampling, we set a new
state-of-the-art on all benchmarks. Our implementation will be publicly
available at https://github.com/ETHRuiGong/PTDiffSeg.

本文研究了将扩散模型预训练的特征表示用于语义分割的能力，发现这种方法可以在新的领域中实现非常出色的泛化能力，并提出了基于输入提示的跨域性能增强策略，包括场景提示、提示随机化、无监督学习的场景提示等，以及测试时域适应的方法，可以在不需要其他复杂技术的情况下，在四个合成到真实、清晰到恶劣天气数据集上实现最新的最优性能。

跨领域语义分割的激励扩散表示

Prompting Diffusion Representations for Cross-Domain Semantic  Segmentation

Anomaly detection aims to separate anomalies from normal samples, and the
pretrained network is promising for anomaly detection. However, adapting the
pretrained features would be confronted with the risk of pattern collapse when
finetuning on one-class training data. In this paper, we propose an anomaly
detection framework called constrained adaptive projection with pretrained
features (CAP). Combined with pretrained features, a simple linear projection
head applied on a specific input and its k most similar pretrained normal
representations is designed for feature adaptation, and a reformed
self-attention is leveraged to mine the inner-relationship among one-class
semantic features. A loss function is proposed to avoid potential pattern
collapse. Concretely, it considers the similarity between a specific data and
its corresponding adaptive normal representation, and incorporates a constraint
term slightly aligning pretrained and adaptive spaces. Our method achieves
state-ofthe-art anomaly detection performance on semantic anomaly detection and
sensory anomaly detection benchmarks including 96.5% AUROC on CIFAR- 100
dataset, 97.0% AUROC on CIFAR-10 dataset and 89.9% AUROC on MvTec dataset.

本文提出一种基于预训练特征的异常检测框架 (CAP)，该框架通过线性投影头将输入数据和最相似的预训练正常数据匹配，并采用改进的自注意力机制挖掘单类语义特征之间的内在关系，以避免潜在的模式折叠问题，实现了语义异常检测和感知异常检测基准上的最优性能，如 CIFAR-100 数据集上的 96.5％ AUROC，CIFAR-10 数据集上的 97.0％ AUROC 和 MvTec 数据集上的 89.9％ AUROC。

基于预训练特征的受限自适应投影异常检测

Constrained Adaptive Projection with Pretrained Features for Anomaly  Detection

Anomaly detection methods require high-quality features. In recent years, the
anomaly detection community has attempted to obtain better features using
advances in deep self-supervised feature learning. Surprisingly, a very
promising direction, using pretrained deep features, has been mostly
overlooked. In this paper, we first empirically establish the perhaps expected,
but unreported result, that combining pretrained features with simple anomaly
detection and segmentation methods convincingly outperforms, much more complex,
state-of-the-art methods.
In order to obtain further performance gains in anomaly detection, we adapt
pretrained features to the target distribution. Although transfer learning
methods are well established in multi-class classification problems, the
one-class classification (OCC) setting is not as well explored. It turns out
that naive adaptation methods, which typically work well in supervised
learning, often result in catastrophic collapse (feature deterioration) and
reduce performance in OCC settings. A popular OCC method, DeepSVDD, advocates
using specialized architectures, but this limits the adaptation performance
gain. We propose two methods for combating collapse: i) a variant of early
stopping that dynamically learns the stopping iteration ii) elastic
regularization inspired by continual learning. Our method, PANDA, outperforms
the state-of-the-art in the OCC, outlier exposure and anomaly segmentation
settings by large margins.

本文研究在异常检测领域中采用预训练特征的方法，并通过对特征的动态适应以及弹性正则化等方式提高异常检测、异常暴露和异常分割的性能，取得了领先于现有方法的效果。