This paper serves to introduce the Align, Minimize and Diversify (AMD)
method, a Source-Free Unsupervised Domain Adaptation approach for Handwritten
Text Recognition (HTR). This framework decouples the adaptation process from
the source data, thus not only sidestepping the resource-intensive retraining
process but also making it possible to leverage the wealth of pre-trained
knowledge encoded in modern Deep Learning architectures. Our method explicitly
eliminates the need to revisit the source data during adaptation by
incorporating three distinct regularization terms: the Align term, which
reduces the feature distribution discrepancy between source and target data,
ensuring the transferability of the pre-trained representation; the Minimize
term, which encourages the model to make assertive predictions, pushing the
outputs towards one-hot-like distributions in order to minimize prediction
uncertainty, and finally, the Diversify term, which safeguards against the
degeneracy in predictions by promoting varied and distinctive sequences
throughout the target data, preventing informational collapse. Experimental
results from several benchmarks demonstrated the effectiveness and robustness
of AMD, showing it to be competitive and often outperforming DA methods in HTR.

该论文介绍了一种用于手写文本识别的 Align, Minimize 和 Diversify（AMD）方法，它是一种无源无监督域自适应方法。通过解耦适应过程和源数据，这种方法不仅避免了资源密集型的重新训练过程，还能够利用现代深度学习架构中编码的丰富预训练知识。通过引入三个独立的正则化项，即对齐项、最小化项和多样化项，我们的方法明确消除了在适应过程中需要重新访问源数据的需求，保证了预训练表示的可迁移性，最小化预测的不确定性，同时通过促进目标数据中的多样性和独特序列，防止信息崩溃。多个基准实验结果证明了 AMD 的有效性和鲁棒性，在手写文本识别中竞争力强，常常胜过之前的域自适应方法。

对齐、最小化和多样化：一种基于源自由的无监督领域自适应方法用于手写文字识别

Align, Minimize and Diversify: A Source-Free Unsupervised Domain  Adaptation Method for Handwritten Text Recognition

Pre-trained representation is one of the key elements in the success of
modern deep learning. However, existing works on continual learning methods
have mostly focused on learning models incrementally from scratch. In this
paper, we explore an alternative framework to incremental learning where we
continually fine-tune the model from a pre-trained representation. Our method
takes advantage of linearization technique of a pre-trained neural network for
simple and effective continual learning. We show that this allows us to design
a linear model where quadratic parameter regularization method is placed as the
optimal continual learning policy, and at the same time enjoying the high
performance of neural networks. We also show that the proposed algorithm
enables parameter regularization methods to be applied to class-incremental
problems. Additionally, we provide a theoretical reason why the existing
parameter-space regularization algorithms such as EWC underperform on neural
networks trained with cross-entropy loss. We show that the proposed method can
prevent forgetting while achieving high continual fine-tuning performance on
image classification tasks. To show that our method can be applied to general
continual learning settings, we evaluate our method in data-incremental,
task-incremental, and class-incremental learning problems.

本文提出了一种基于预训练表示的持续微调模型的方法，其中使用线性化技术和二次参数正则化策略，可以在图像分类任务中实现高持续微调性能并防止遗忘，并可应用于数据、任务和类增量学习问题。

DLCFT：深度线性连续微调在通用增量学习中的应用

DLCFT: Deep Linear Continual Fine-Tuning for General Incremental Learning

Deep anomaly detection methods learn representations that separate between
normal and anomalous images. Although self-supervised representation learning
is commonly used, small dataset sizes limit its effectiveness. It was
previously shown that utilizing external, generic datasets (e.g. ImageNet
classification) can significantly improve anomaly detection performance. One
approach is outlier exposure, which fails when the external datasets do not
resemble the anomalies. We take the approach of transferring representations
pre-trained on external datasets for anomaly detection. Anomaly detection
performance can be significantly improved by fine-tuning the pre-trained
representations on the normal training images. In this paper, we first
demonstrate and analyze that contrastive learning, the most popular
self-supervised learning paradigm cannot be naively applied to pre-trained
features. The reason is that pre-trained feature initialization causes poor
conditioning for standard contrastive objectives, resulting in bad optimization
dynamics. Based on our analysis, we provide a modified contrastive objective,
the Mean-Shifted Contrastive Loss. Our method is highly effective and achieves
a new state-of-the-art anomaly detection performance including $98.6\%$ ROC-AUC
on the CIFAR-10 dataset.

本文提出一种新的深度异常检测方法，利用预训练的表示在外部数据集上进行对比学习，并基于分析提出改进的对比学习目标函数 Mean-Shifted Contrastive Loss，该方法可以显著提高异常检测性能。实验结果表明，与现有方法相比，在 CIFAR-10 数据集上实现了 98.6％的 ROC-AUC 表现，达到了最新的技术水平。