Adversarial contrastive learning (ACL), without requiring labels,
incorporates adversarial data with standard contrastive learning (SCL) and
outputs a robust representation which is generalizable and resistant to
adversarial attacks and common corruptions. The style-independence property of
representations has been validated to be beneficial in improving robustness
transferability. Standard invariant regularization (SIR) has been proposed to
make the learned representations via SCL to be independent of the style
factors. However, how to equip robust representations learned via ACL with the
style-independence property is still unclear so far. To this end, we leverage
the technique of causal reasoning to propose an adversarial invariant
regularization (AIR) that enforces robust representations learned via ACL to be
style-independent. Then, we enhance ACL using invariant regularization (IR),
which is a weighted sum of SIR and AIR. Theoretically, we show that AIR
implicitly encourages the prediction of adversarial data and consistency
between adversarial and natural data to be independent of data augmentations.
We also theoretically demonstrate that the style-independence property of
robust representation learned via ACL still holds in downstream tasks,
providing generalization guarantees. Empirically, our comprehensive
experimental results corroborate that IR can significantly improve the
performance of ACL and its variants on various datasets.

本论文通过采用因果推理技术提出了一种对抗不变正则化（AIR）的方法，使 ACL 学习到的强鲁棒表示具有独立于样式的特性，并结合 SIR 形成一个加权和为 IR 的方法，理论上证明 AIR 可以鼓励对抗数据的预测和一致性与自然数据不受数据增强的影响而呈现独立性，并表明 ACL 学习到的鲁棒表示的 style-independence 属性在下游任务中仍然成立。