This paper makes two contributions to the field of text-based patent
similarity. First, it compares the performance of different kinds of
patent-specific pretrained embedding models, namely static word embeddings
(such as word2vec and doc2vec models) and contextual word embeddings (such as
transformers based models), on the task of patent similarity calculation.
Second, it compares specifically the performance of Sentence Transformers
(SBERT) architectures with different training phases on the patent similarity
task. To assess the models' performance, we use information about patent
interferences, a phenomenon in which two or more patent claims belonging to
different patent applications are proven to be overlapping by patent examiners.
Therefore, we use these interferences cases as a proxy for maximum similarity
between two patents, treating them as ground-truth to evaluate the performance
of the different embedding models. Our results point out that, first, Patent
SBERT-adapt-ub, the domain adaptation of the pretrained Sentence Transformer
architecture proposed in this research, outperforms the current
state-of-the-art in patent similarity. Second, they show that, in some cases,
large static models performances are still comparable to contextual ones when
trained on extensive data; thus, we believe that the superiority in the
performance of contextual embeddings may not be related to the actual
architecture but rather to the way the training phase is performed.

该研究在文本专利相似性领域有两方面的贡献：首先，它比较了不同类型的专利特定预训练嵌入模型，包括静态词嵌入（如 word2vec 和 doc2vec 模型）和上下文词嵌入（如基于 transformers 的模型），在专利相似性计算任务上的性能；其次，它比较了 Sentence Transformers（SBERT）架构在专利相似性任务上不同训练阶段的性能。结果表明，该研究提出的专利 SBERT-adapt-ub，即预训练 Sentence Transformer 架构的领域自适应，优于当前专利相似性的最新技术水平。其次，研究结果显示，在某些情况下，大型静态模型在训练大量数据时仍可以与上下文模型相媲美；因此，我们认为上下文嵌入在性能上的优势可能与实际架构无关，而是与训练阶段的方式有关。