Test-time augmentation -- the aggregation of predictions across transformed
examples of test inputs -- is an established technique to improve the
performance of image classification models. Importantly, TTA can be used to
improve model performance post-hoc, without additional training. Although
test-time augmentation (TTA) can be applied to any data modality, it has seen
limited adoption in NLP due in part to the difficulty of identifying
label-preserving transformations. In this paper, we present augmentation
policies that yield significant accuracy improvements with language models. A
key finding is that augmentation policy design -- for instance, the number of
samples generated from a single, non-deterministic augmentation -- has a
considerable impact on the benefit of TTA. Experiments across a binary
classification task and dataset show that test-time augmentation can deliver
consistent improvements over current state-of-the-art approaches.

本文提出了一种改进的测试时间增强技术并设计出相应的增强策略，结果表明用于语言模型上的测试时间增强可以显著提高模型的准确性。

通过测试时间增强来改进文本分类

Improved Text Classification via Test-Time Augmentation

Despite the large augmentation family, only a few cherry-picked robust
augmentation policies are beneficial to self-supervised image representation
learning. In this paper, we propose a directional self-supervised learning
paradigm (DSSL), which is compatible with significantly more augmentations.
Specifically, we adapt heavy augmentation policies after the views lightly
augmented by standard augmentations, to generate harder view (HV). HV usually
has a higher deviation from the original image than the lightly augmented
standard view (SV). Unlike previous methods equally pairing all augmented views
to symmetrically maximize their similarities, DSSL treats augmented views of
the same instance as a partially ordered set (with directions as
SV$\leftrightarrow $SV, SV$\leftarrow$HV), and then equips a directional
objective function respecting to the derived relationships among views. DSSL
can be easily implemented with a few lines of codes and is highly flexible to
popular self-supervised learning frameworks, including SimCLR, SimSiam, BYOL.
Extensive experimental results on CIFAR and ImageNet demonstrated that DSSL can
stably improve various baselines with compatibility to a wider range of
augmentations.

本文提出了一种方向性自我监督学习模式，采用重度增强策略生成更难学习的视角并将其与轻度增强视图进行匹配，建立了基于方向性的目标函数来提高模型的性能表现；实验结果表明，该模式能够有效训练出更具泛化性的模型。

重型图片增强的定向自监督学习

Directional Self-supervised Learning for Heavy Image Augmentations

A common practice in unsupervised representation learning is to use labeled
data to evaluate the quality of the learned representations. This supervised
evaluation is then used to guide critical aspects of the training process such
as selecting the data augmentation policy. However, guiding an unsupervised
training process through supervised evaluations is not possible for real-world
data that does not actually contain labels (which may be the case, for example,
in privacy sensitive fields such as medical imaging). Therefore, in this work
we show that evaluating the learned representations with a self-supervised
image rotation task is highly correlated with a standard set of supervised
evaluations (rank correlation $> 0.94$). We establish this correlation across
hundreds of augmentation policies, training settings, and network architectures
and provide an algorithm (SelfAugment) to automatically and efficiently select
augmentation policies without using supervised evaluations. Despite not using
any labeled data, the learned augmentation policies perform comparably with
augmentation policies that were determined using exhaustive supervised
evaluations.

本文通过自监督的方式，解决了在无监督学习中选取数据扩增策略的问题，同时验证了自监督学习的准确性。