We marry two powerful ideas: deep representation learning for visual
recognition and language understanding, and symbolic program execution for
reasoning. Our neural-symbolic visual question answering (NS-VQA) system first
recovers a structural scene representation from the image and a program trace
from the question. It then executes the program on the scene representation to
obtain an answer. Incorporating symbolic structure as prior knowledge offers
three unique advantages. First, executing programs on a symbolic space is more
robust to long program traces; our model can solve complex reasoning tasks
better, achieving an accuracy of 99.8% on the CLEVR dataset. Second, the model
is more data- and memory-efficient: it performs well after learning on a small
number of training data; it can also encode an image into a compact
representation, requiring less storage than existing methods for offline
question answering. Third, symbolic program execution offers full transparency
to the reasoning process; we are thus able to interpret and diagnose each
execution step.

该论文介绍了神经符号视觉问答系统，结合深度表示学习和符号程序执行，具有更好的数据和内存效率以及对推理过程的全透明度。