This study introduces a simple yet effective method for identifying similar
data points across non-free text domains, such as tabular and image data, using
Large Language Models (LLMs). Our two-step approach involves data point
summarization and hidden state extraction. Initially, data is condensed via
summarization using an LLM, reducing complexity and highlighting essential
information in sentences. Subsequently, the summarization sentences are fed
through another LLM to extract hidden states, serving as compact, feature-rich
representations. This approach leverages the advanced comprehension and
generative capabilities of LLMs, offering a scalable and efficient strategy for
similarity identification across diverse datasets. We demonstrate the
effectiveness of our method in identifying similar data points on multiple
datasets. Additionally, our approach enables non-technical domain experts, such
as fraud investigators or marketing operators, to quickly identify similar data
points tailored to specific scenarios, demonstrating its utility in practical
applications. In general, our results open new avenues for leveraging LLMs in
data analysis across various domains.

本研究提出了一种简单而有效的方法，利用大语言模型（LLM）在非自由文领域（如表格和图像数据）中识别相似数据点。我们的两步方法涉及数据点摘要和隐藏状态提取。首先，通过使用 LLM 进行摘要，将数据压缩，降低复杂性并突出句子中的关键信息。随后，将摘要句送入另一个 LLM 中提取隐藏状态，作为紧凑、特征丰富的表示形式。此方法利用 LLM 的高级理解和生成能力，提供了一种可扩展且高效的相似性识别策略，适用于各种数据集。我们通过在多个数据集上演示了我们方法在识别相似数据点方面的有效性。此外，我们的方法使非技术领域专家（如欺诈调查员或营销运营人员）能够快速识别适用于特定场景的相似数据点，展示了其在实际应用中的实用性。总体上，我们的结果为在各个领域中利用 LLM 进行数据分析开辟了新的道路。