We attribute the vulnerability of natural language processing models to the
fact that similar inputs are converted to dissimilar representations in the
embedding space, leading to inconsistent outputs, and we propose a novel robust
training method, termed Fast Triplet Metric Learning (FTML). Specifically, we
argue that the original sample should have similar representation with its
adversarial counterparts and distinguish its representation from other samples
for better robustness. To this end, we adopt the triplet metric learning into
the standard training to pull words closer to their positive samples (i.e.,
synonyms) and push away their negative samples (i.e., non-synonyms) in the
embedding space. Extensive experiments demonstrate that FTML can significantly
promote the model robustness against various advanced adversarial attacks while
keeping competitive classification accuracy on original samples. Besides, our
method is efficient as it only needs to adjust the embedding and introduces
very little overhead on the standard training. Our work shows great potential
of improving the textual robustness through robust word embedding.

本研究提出了一种新的鲁棒训练方法，即 Fast Triplet Metric Learning (FTML)，通过采用三元组度量学习来让相似的样本在嵌入空间中的表示更接近，并与其敌对样本有类似的表示，从而提高自然语言处理模型在对抗攻击下的鲁棒性。实验证明，该方法不仅有效地提高了模型的鲁棒性，而且效率高，可在标准训练中引入很少的开销，具有很大的潜力来提高文本的鲁棒性。