With the advances in large scale vision-and-language models (VLMs) it is of
interest to assess their performance on various visual reasoning tasks such as
counting, referring expressions and general visual question answering. The
focus of this work is to study the ability of these models to understanding
spatial relations. Previously, this has been tackled using image-text matching
(Liu, Emerson, and Collier 2022) or visual question answering task, both
showing poor performance and a large gap compared to human performance. To
better understand the gap, we present fine-grained compositional grounding of
spatial relationships and propose a bottom up approach for ranking spatial
clauses and evaluating the performance of spatial relationship reasoning task.
We propose to combine the evidence from grounding noun phrases corresponding to
objects and their locations to compute the final rank of the spatial clause. We
demonstrate the approach on representative vision-language models (Tan and
Bansal 2019; Gupta et al. 2022; Kamath et al. 2021) and compare and highlight
their abilities to reason about spatial relationships.

利用大规模视觉语言模型评估其在不同视觉推理任务中的性能，特别是在理解空间关系方面的能力的研究。通过细粒度的组合性先验来解决空间关系推理任务的评估和排名，结合物体及其位置的核心语义来计算空间子句的最终评分，并比较不同视觉语言模型在空间关系推理方面的能力。