We study semantic collisions: texts that are semantically unrelated but
judged as similar by NLP models. We develop gradient-based approaches for
generating semantic collisions and demonstrate that state-of-the-art models for
many tasks which rely on analyzing the meaning and similarity of texts--
including paraphrase identification, document retrieval, response suggestion,
and extractive summarization-- are vulnerable to semantic collisions. For
example, given a target query, inserting a crafted collision into an irrelevant
document can shift its retrieval rank from 1000 to top 3. We show how to
generate semantic collisions that evade perplexity-based filtering and discuss
other potential mitigations. Our code is available at
this https URL

本论文研究语义碰撞，即在语义上不相关但被 NLP 模型认为相似的文本。我们开发了基于梯度的方法用于生成语义碰撞，并证明了许多任务的最新模型，这些任务依赖于分析文本的含义和相似性，包括释义识别、文档检索、响应建议和提取式摘要等方面的模型都容易受到语义冲突的影响。例如，针对一个目标查询，将一个精心制作的语义碰撞插入到一个无关的文档中可以将其检索排名从 1000 上升到前三。我们展示了如何生成避开清晰度过滤器的语义冲突，并讨论了其他可能的缓解方法。