Can a Large Language Model (LLM) solve simple abstract reasoning problems? We
explore this broad question through a systematic analysis of GPT on the
Abstraction and Reasoning Corpus (ARC), a representative benchmark of abstract
reasoning ability from limited examples in which solutions require some "core
knowledge" of concepts such as objects, goal states, counting, and basic
geometry. GPT-4 solves only 13/50 of the most straightforward ARC tasks when
using textual encodings for their two-dimensional input-output grids. Our
failure analysis reveals that GPT-4's capacity to identify objects and reason
about them is significantly influenced by the sequential nature of the text
that represents an object within a text encoding of a task. To test this
hypothesis, we design a new benchmark, the 1D-ARC, which consists of
one-dimensional (array-like) tasks that are more conducive to GPT-based
reasoning, and where it indeed performs better than on the (2D) ARC. To
alleviate this issue, we propose an object-based representation that is
obtained through an external tool, resulting in nearly doubling the performance
on solved ARC tasks and near-perfect scores on the easier 1D-ARC. Although the
state-of-the-art GPT-4 is unable to "reason" perfectly within non-language
domains such as the 1D-ARC or a simple ARC subset, our study reveals that the
use of object-based representations can significantly improve its reasoning
ability. Visualizations, GPT logs, and data are available at
this https URL

本文探讨了大型语言模型在抽象推理问题中的解决能力，提出了基于物体表示的解决方案，解决了 ARC 数据集中的 13 个问题，并在 1D-ARC 数据集上取得了接近完美的成绩。

LLMs 与抽象推理数据集：成功、失败及基于对象表示的重要性

LLMs and the Abstraction and Reasoning Corpus: Successes, Failures, and  the Importance of Object-based Representations

The world consists of objects: distinct entities possessing independent
properties and dynamics. For agents to interact with the world intelligently,
they must translate sensory inputs into the bound-together features that
describe each object. These object-based representations form a natural basis
for planning behavior. Active inference (AIF) is an influential unifying
account of perception and action, but existing AIF models have not leveraged
this important inductive bias. To remedy this, we introduce 'object-based
active inference' (OBAI), marrying AIF with recent deep object-based neural
networks. OBAI represents distinct objects with separate variational beliefs,
and uses selective attention to route inputs to their corresponding object
slots. Object representations are endowed with independent action-based
dynamics. The dynamics and generative model are learned from experience with a
simple environment (active multi-dSprites). We show that OBAI learns to
correctly segment the action-perturbed objects from video input, and to
manipulate these objects towards arbitrary goals.

本研究介绍了一种结合了近期深度物品类神经网络的主题对象活跃推理（OBAI）的模型，该模型通过独立信念来表征不同物体，使用选择性注意力将输入路由到相应的物体插槽，并学习动态和生成模型，从而正确地将行动扰动物体从视频输入中分割，并将这些物体朝任意目标运动。