Despite recent success, most contrastive self-supervised learning methods are
domain-specific, relying heavily on data augmentation techniques that require
knowledge about a particular domain, such as image cropping and rotation. To
overcome such limitation, we propose a novel domain-agnostic approach to
contrastive learning, named DACL, that is applicable to domains where
invariances, and thus, data augmentation techniques, are not readily available.
Key to our approach is the use of Mixup noise to create similar and dissimilar
examples by mixing data samples differently either at the input or hidden-state
levels. To demonstrate the effectiveness of DACL, we conduct experiments across
various domains such as tabular data, images, and graphs. Our results show that
DACL not only outperforms other domain-agnostic noising methods, such as
Gaussian-noise, but also combines well with domain-specific methods, such as
SimCLR, to improve self-supervised visual representation learning. Finally, we
theoretically analyze our method and show advantages over the Gaussian-noise
based contrastive learning approach.

该研究提出了一种新颖的领域无关对比学习方法 DACl，利用 Mixup 噪声在输入或隐藏状态级别上不同地混合数据样本来创建相似和不相似的示例，并在各种领域进行实验，结果表明 DACl 不仅优于其他领域无关的噪声方法，而且与 SimCLR 等领域特定方法结合，可以提高自监督视觉表示学习。