We introduce CLEVR-Math, a multi-modal math word problems dataset consisting
of simple math word problems involving addition/subtraction, represented partly
by a textual description and partly by an image illustrating the scenario. The
text describes actions performed on the scene that is depicted in the image.
Since the question posed may not be about the scene in the image, but about the
state of the scene before or after the actions are applied, the solver envision
or imagine the state changes due to these actions. Solving these word problems
requires a combination of language, visual and mathematical reasoning. We apply
state-of-the-art neural and neuro-symbolic models for visual question answering
on CLEVR-Math and empirically evaluate their performances. Our results show how
neither method generalise to chains of operations. We discuss the limitations
of the two in addressing the task of multi-modal word problem solving.

通过综合应用语言、视觉和数学推理，本文介绍了一个多模态数学问题数据集 CLEVR-Math，并应用最先进的神经模型和神经符号模型对其进行了实证研究，结果显示这两种方法都无法有效解决多模态数学问题。