Recent research shows that more data and larger models can provide more
accurate solutions to natural language problems requiring reasoning. However,
models can easily fail to provide solutions in unobserved complex input
compositions due to not achieving the level of abstraction required for
generalizability. To alleviate this issue, we propose training the language
models with neuro-symbolic techniques that can exploit the logical rules of
reasoning as constraints and provide additional supervision sources to the
model. Training models to adhere to the regulations of reasoning pushes them to
make more effective abstractions needed for generalizability and transfer
learning. We focus on a challenging problem of spatial reasoning over text. Our
results on various benchmarks using multiple language models confirm our
hypothesis of effective domain transfer based on neuro-symbolic training.

通过使用神经符号技术对语言模型进行培训，可以利用推理的逻辑规则作为约束，并为模型提供额外的监督来源，以弥补由于无法达到所需的抽象水平而在未观察到的复杂输入组合中难以提供解决方案的问题。我们聚焦于文本上的空间推理这一具有挑战性的问题，通过在多个语言模型上的多个基准测试中的结果验证了基于神经符号训练的有效领域转移假设。

神经符号训练用于空间语言推理

Neuro-symbolic Training for Reasoning over Spatial Language

Vision-Language Models (VLMs) are trained on vast amounts of data captured by
humans emulating our understanding of the world. However, known as visual
illusions, human's perception of reality isn't always faithful to the physical
world. This raises a key question: do VLMs have the similar kind of illusions
as humans do, or do they faithfully learn to represent reality? To investigate
this question, we build a dataset containing five types of visual illusions and
formulate four tasks to examine visual illusions in state-of-the-art VLMs. Our
findings have shown that although the overall alignment is low, larger models
are closer to human perception and more susceptible to visual illusions. Our
dataset and initial findings will promote a better understanding of visual
illusions in humans and machines and provide a stepping stone for future
computational models that can better align humans and machines in perceiving
and communicating about the shared visual world. The code and data are
available at this https URL

通过构建包含五种视觉错觉的数据集并制定四项任务，研究发现，虽然整体对齐性较低，但更大的模型更接近人类感知且更容易受到视觉错觉的影响，这将促进对人类和机器中的视觉错觉有更深入的理解，并为未来能够更好地使人类和机器在感知和交流共享的视觉世界方面达到一致的计算模型提供基础。