Contrastive learning has proven to be an effective method for pre-training
models using weakly labeled data in the vision domain. Sentence transformers
are the NLP counterparts to this architecture, and have been growing in
popularity due to their rich and effective sentence representations. Having
effective sentence representations is paramount in multiple tasks, such as
information retrieval, retrieval augmented generation (RAG), and sentence
comparison. Keeping in mind the deployability factor of transformers,
evaluating the robustness of sentence transformers is of utmost importance.
This work focuses on evaluating the robustness of the sentence encoders. We
employ several adversarial attacks to evaluate its robustness. This system uses
character-level attacks in the form of random character substitution,
word-level attacks in the form of synonym replacement, and sentence-level
attacks in the form of intra-sentence word order shuffling. The results of the
experiments strongly undermine the robustness of sentence encoders. The models
produce significantly different predictions as well as embeddings on perturbed
datasets. The accuracy of the models can fall up to 15 percent on perturbed
datasets as compared to unperturbed datasets. Furthermore, the experiments
demonstrate that these embeddings does capture the semantic and syntactic
structure (sentence order) of sentences. However, existing supervised
classification strategies fail to leverage this information, and merely
function as n-gram detectors.

通过对比学习方法评估句子编码器的鲁棒性，使用字符级、词级和句级攻击检验了模型的稳健性，结果显示模型在扰动数据集上的准确率可下降高达 15％，且当前的分类策略未能充分利用语义和句法结构信息。