In patent prosecution, image-based retrieval systems for identifying
similarities between current patent images and prior art are pivotal to ensure
the novelty and non-obviousness of patent applications. Despite their growing
popularity in recent years, existing attempts, while effective at recognizing
images within the same patent, fail to deliver practical value due to their
limited generalizability in retrieving relevant prior art. Moreover, this task
inherently involves the challenges posed by the abstract visual features of
patent images, the skewed distribution of image classifications, and the
semantic information of image descriptions. Therefore, we propose a
language-informed, distribution-aware multimodal approach to patent image
feature learning, which enriches the semantic understanding of patent image by
integrating Large Language Models and improves the performance of
underrepresented classes with our proposed distribution-aware contrastive
losses. Extensive experiments on DeepPatent2 dataset show that our proposed
method achieves state-of-the-art or comparable performance in image-based
patent retrieval with mAP +53.3%, Recall@10 +41.8%, and MRR@10 +51.9%.
Furthermore, through an in-depth user analysis, we explore our model in aiding
patent professionals in their image retrieval efforts, highlighting the model's
real-world applicability and effectiveness.

在专利审查中，基于图像的检索系统对于识别当前专利图像与现有技术之间的相似性至关重要，以确保专利申请的新颖性和非显而易见性。我们提出了一种基于语言的、分布感知的多模态专利图像特征学习方法，通过集成大型语言模型，丰富专利图像的语义理解，并通过我们提出的分布感知对比损失来提高在代表性类别中的性能。在 DeepPatent2 数据集上进行的大量实验表明，我们提出的方法在基于图像的专利检索方面取得了最先进或可比较的性能，平均准确率提高了 53.3％，前 10 个检索结果的召回率提高了 41.8％，前 10 个检索结果的 MRR 提高了 51.9％。此外，通过深入用户分析，我们探索了我们的模型在帮助专利专业人员进行图像检索方面的作用，凸显了该模型的实际适用性和效果。

大型语言模型在专利图像检索中的应用

Large Language Model Informed Patent Image Retrieval

Patent examiners need to solve a complex information retrieval task when they
assess the novelty and inventive step of claims made in a patent application.
Given a claim, they search for prior art, which comprises all relevant publicly
available information. This time-consuming task requires a deep understanding
of the respective technical domain and the patent-domain-specific language. For
these reasons, we address the computer-assisted search for prior art by
creating a training dataset for supervised machine learning called PatentMatch.
It contains pairs of claims from patent applications and semantically
corresponding text passages of different degrees from cited patent documents.
Each pair has been labeled by technically-skilled patent examiners from the
European Patent Office. Accordingly, the label indicates the degree of semantic
correspondence (matching), i.e., whether the text passage is prejudicial to the
novelty of the claimed invention or not. Preliminary experiments using a
baseline system show that PatentMatch can indeed be used for training a binary
text pair classifier on this challenging information retrieval task. The
dataset is available online: this https URL

本文介绍了一个名为 PatentMatch 的机器学习训练数据集，用于协助专利审查员快速检索相关先前公开信息，其数据集包含有标签的专利申请索赔对和语义上对应的先前公开信息，可用于解决语言障碍和技术域障碍，初步实验结果显示该数据集可用于解决这一复杂的信息检索任务。