The Connections puzzle published each day by the New York Times tasks players
with dividing a bank of sixteen words into four groups of four words that each
relate to a common theme. Solving the puzzle requires both common linguistic
knowledge (i.e. definitions and typical usage) as well as, in many cases,
lateral or abstract thinking. This is because the four categories ascend in
complexity, with the most challenging category often requiring thinking about
words in uncommon ways or as parts of larger phrases. We investigate the
capacity for automated AI systems to play Connections and explore the game's
potential as an automated benchmark for abstract reasoning and a way to measure
the semantic information encoded by data-driven linguistic systems. In
particular, we study both a sentence-embedding baseline and modern large
language models (LLMs). We report their accuracy on the task, measure the
impacts of chain-of-thought prompting, and discuss their failure modes.
Overall, we find that the Connections task is challenging yet feasible, and a
strong test-bed for future work.

通过研究自动化 AI 系统在 Connections 谜题中的效果，该研究探讨了该游戏作为抽象推理的自动化基准测试和衡量数据驱动语言系统中编码的语义信息的潜力。研究发现 Connections 谜题具有挑战性和可行性，是未来工作的重要测试平台。