We propose a new class of probabilistic neural-symbolic models, that have symbolic functional programs as a latent, stochastic variable. Instantiated in the context of visual question answering, our probabilistic formulation offers two key conceptual advantages over prior neural-symbolic models for VQA. Firstly, the programs generated by our model are more understandable while requiring lesser number of teaching examples. Secondly, we show that one can pose counterfactual scenarios to the model, to probe its beliefs on the programs that could lead to a specified answer given an image. Our results on the CLEVR and SHAPES datasets verify our hypotheses, showing that the model gets better program (and answer) prediction accuracy even in the low data regime, and allows one to probe the coherence and consistency of reasoning performed.

本研究提出了一种新的概率神经符号模型，其拥有作为潜在随机变量的符号功能程序，并在视觉问题回答的背景下实例化，相比之前的神经符号模型，我们的模型提供了两个关键概念上的优势，第一，我们的模型生成的程序更易理解，需要较少的教学示例； 第二，我们展示了我们可以向该模型提出反事实情景，来探究其对于给定图像可以导致特定答案的程序的信念，我们对CLEVR和SHAPES数据集的结果验证了我们的假设，在低数据情况下，模型具有更好的方案（和答案）预测准确性，并允许人们探究所执行推理的连贯性和一致性。

可解释性视觉问答的概率神经符号模型