A fundamental challenge in artificial intelligence is learning useful
representations of data that yield good performance on a downstream task,
without overfitting to spurious input features. Extracting such task-relevant
predictive information is particularly difficult for real-world datasets. In
this work, we propose Contrastive Input Morphing (CIM), a representation
learning framework that learns input-space transformations of the data to
mitigate the effect of irrelevant input features on downstream performance. Our
method leverages a perceptual similarity metric via a triplet loss to ensure
that the transformation preserves task-relevant information.Empirically, we
demonstrate the efficacy of our approach on tasks which typically suffer from
the presence of spurious correlations: classification with nuisance
information, out-of-distribution generalization, and preservation of subgroup
accuracies. We additionally show that CIM is complementary to other mutual
information-based representation learning techniques, and demonstrate that it
improves the performance of variational information bottleneck (VIB) when used
together.

本文提出了一种学习输入空间变换的表示学习框架，通过感知相似度度量的三元损失确保变换保留任务相关信息，并在分类，越域泛化和子组准确性保持等任务上取得了良好效果。

基于感知相似度度量的稳健表征学习

Robust Representation Learning via Perceptual Similarity Metrics

This paper proposes a new end-to-end trainable model for lossy image
compression, which includes several novel components. The method incorporates
1) an adequate perceptual similarity metric; 2) saliency in the images; 3) a
hierarchical auto-regressive model. This paper demonstrates that the popularly
used evaluations metrics such as MS-SSIM and PSNR are inadequate for judging
the performance of image compression techniques as they do not align with the
human perception of similarity. Alternatively, a new metric is proposed, which
is learned on perceptual similarity data specific to image compression. The
proposed compression model incorporates the salient regions and optimizes on
the proposed perceptual similarity metric. The model not only generates images
which are visually better but also gives superior performance for subsequent
computer vision tasks such as object detection and segmentation when compared
to existing engineered or learned compression techniques.

本文提出了一种新的端到端可训练模型，用于有损图像压缩。该方法包括几个新颖的组件，其中包括充分的感知相似度度量、图像中的显着性和分层自回归模型。该模型不仅生成视觉效果更好的图像，而且在后续计算机视觉任务（如目标检测和分割）中也具有优越的性能。

基于显著性驱动的感知图像压缩

Saliency Driven Perceptual Image Compression

It has been recently shown that the hidden variables of convolutional neural
networks make for an efficient perceptual similarity metric that accurately
predicts human judgment on relative image similarity assessment. First, we show
that such learned perceptual similarity metrics (LPIPS) are susceptible to
adversarial attacks that dramatically contradict human visual similarity
judgment. While this is not surprising in light of neural networks' well-known
weakness to adversarial perturbations, we proceed to show that self-ensembling
with an infinite family of random transformations of the input --- a technique
known not to render classification networks robust --- is enough to turn the
metric robust against attack, while retaining predictive power on human
judgments. Finally, we study the geometry imposed by our our novel
self-ensembled metric (E-LPIPS) on the space of natural images. We find
evidence of "perceptual convexity" by showing that convex combinations of
similar-looking images retain appearance, and that discrete geodesics yield
meaningful frame interpolation and texture morphing, all without explicit
correspondences.

本文研究了卷积神经网络的隐藏变量作为感知相似度度量的有效性，发现自学习的感知相似度度量（LPIPS）容易受到对抗样本的攻击，但无限家族随机变换的自增强技术可以使度量鲁棒性提高，同时保持对人类判断的预测能力，并发现自增强度量空间中的 “感知凸性” 现象，即相似图像的凸组合保留外观，离散测地线产生有意义的帧插值和纹理变形。