This paper makes two contributions to the field of text-based patent
similarity. First, it compares the performance of different kinds of
patent-specific pretrained embedding models, namely static word embeddings
(such as word2vec and doc2vec models) and contextual word embeddings (such as
transformers based models), on the task of patent similarity calculation.
Second, it compares specifically the performance of Sentence Transformers
(SBERT) architectures with different training phases on the patent similarity
task. To assess the models' performance, we use information about patent
interferences, a phenomenon in which two or more patent claims belonging to
different patent applications are proven to be overlapping by patent examiners.
Therefore, we use these interferences cases as a proxy for maximum similarity
between two patents, treating them as ground-truth to evaluate the performance
of the different embedding models. Our results point out that, first, Patent
SBERT-adapt-ub, the domain adaptation of the pretrained Sentence Transformer
architecture proposed in this research, outperforms the current
state-of-the-art in patent similarity. Second, they show that, in some cases,
large static models performances are still comparable to contextual ones when
trained on extensive data; thus, we believe that the superiority in the
performance of contextual embeddings may not be related to the actual
architecture but rather to the way the training phase is performed.

该研究在文本专利相似性领域有两方面的贡献：首先，它比较了不同类型的专利特定预训练嵌入模型，包括静态词嵌入（如 word2vec 和 doc2vec 模型）和上下文词嵌入（如基于 transformers 的模型），在专利相似性计算任务上的性能；其次，它比较了 Sentence Transformers（SBERT）架构在专利相似性任务上不同训练阶段的性能。结果表明，该研究提出的专利 SBERT-adapt-ub，即预训练 Sentence Transformer 架构的领域自适应，优于当前专利相似性的最新技术水平。其次，研究结果显示，在某些情况下，大型静态模型在训练大量数据时仍可以与上下文模型相媲美；因此，我们认为上下文嵌入在性能上的优势可能与实际架构无关，而是与训练阶段的方式有关。

专利相似性的嵌入模型比较分析

A comparative analysis of embedding models for patent similarity

Consistency models, which were proposed to mitigate the high computational
overhead during the sampling phase of diffusion models, facilitate single-step
sampling while attaining state-of-the-art empirical performance. When
integrated into the training phase, consistency models attempt to train a
sequence of consistency functions capable of mapping any point at any time step
of the diffusion process to its starting point. Despite the empirical success,
a comprehensive theoretical understanding of consistency training remains
elusive. This paper takes a first step towards establishing theoretical
underpinnings for consistency models. We demonstrate that, in order to generate
samples within $\varepsilon$ proximity to the target in distribution (measured
by some Wasserstein metric), it suffices for the number of steps in consistency
learning to exceed the order of $d^{5/2}/\varepsilon$, with $d$ the data
dimension. Our theory offers rigorous insights into the validity and efficacy
of consistency models, illuminating their utility in downstream inference
tasks.

一步采样的连贯性模型在训练阶段中具有良好的实证效果，通过训练一系列连贯性函数，能够将扩散过程的任何时间步骤的任意点映射到其起始点，其理论基础表明，为了生成与目标分布在 ε 接近程度内的样本，连贯性学习的步骤数应超过 d^5/2/ε，其中 d 为数据维度，这一理论为连贯性模型的有效性和实用性提供了深入洞察，从而启示其在下游推理任务中的实用性。