Analogy-making is central to human cognition, allowing us to adapt to novel
situations -- an ability that current AI systems still lack. Most analogy
datasets today focus on simple analogies (e.g., word analogies); datasets
including complex types of analogies are typically manually curated and very
small. We believe that this holds back progress in computational analogy. In
this work, we design a data generation pipeline, ParallelPARC (Parallel
Paragraph Creator) leveraging state-of-the-art Large Language Models (LLMs) to
create complex, paragraph-based analogies, as well as distractors, both simple
and challenging. We demonstrate our pipeline and create ProPara-Logy, a dataset
of analogies between scientific processes. We publish a gold-set, validated by
humans, and a silver-set, generated automatically. We test LLMs' and humans'
analogy recognition in binary and multiple-choice settings, and found that
humans outperform the best models (~13% gap) after a light supervision. We
demonstrate that our silver-set is useful for training models. Lastly, we show
challenging distractors confuse LLMs, but not humans. We hope our pipeline will
encourage research in this emerging field.

通过设计数据生成流水线，利用最先进的大型语言模型创建复杂的基于段落的类比，我们创建了 ProPara-Logy 数据集，用于科学过程之间的类比；在二进制和多选题设置中，我们测试了 LLMs 和人类的类比识别，发现在轻量级监督之后人类的性能优于最佳模型（13％差异），同时证明我们的银标准集对于训练模型很有用；最后，我们展示了具有挑战性的干扰物混淆 LLMs 但不混淆人类，希望我们的流水线能够鼓励该新兴领域的研究。

ParallelPARC：生成自然语言类比的可扩展管道

ParallelPARC: A Scalable Pipeline for Generating Natural-Language  Analogies

Tracking entities in procedural language requires understanding the
transformations arising from actions on entities as well as those entities'
interactions. While self-attention-based pre-trained language encoders like GPT
and BERT have been successfully applied across a range of natural language
understanding tasks, their ability to handle the nuances of procedural texts is
still untested. In this paper, we explore the use of pre-trained transformer
networks for entity tracking tasks in procedural text. First, we test standard
lightweight approaches for prediction with pre-trained transformers, and find
that these approaches underperform even simple baselines. We show that much
stronger results can be attained by restructuring the input to guide the
transformer model to focus on a particular entity. Second, we assess the degree
to which transformer networks capture the process dynamics, investigating such
factors as merged entities and oblique entity references. On two different
tasks, ingredient detection in recipes and QA over scientific processes, we
achieve state-of-the-art results, but our models still largely attend to
shallow context clues and do not form complex representations of intermediate
entity or process state.

本研究探讨了在过程性文本中使用预训练的 transformer 网络跟踪实体的能力，发现通过调整输入结构，可以取得更好的实体跟踪结果，在配方中实现了材料检测以及在科学过程中进行了问答，达到了最先进的结果，但我们的模型仍然主要关注表面上下文线索，没有形成中间实体或进程状态的复杂表示。