Understanding multimodal perception for embodied AI is an open question
because such inputs may contain highly complementary as well as redundant
information for the task. A relevant direction for multimodal policies is
understanding the global trends of each modality at the fusion layer. To this
end, we disentangle the attributions for visual, language, and previous action
inputs across different policies trained on the ALFRED dataset. Attribution
analysis can be utilized to rank and group the failure scenarios, investigate
modeling and dataset biases, and critically analyze multimodal EAI policies for
robustness and user trust before deployment. We present MAEA, a framework to
compute global attributions per modality of any differentiable policy. In
addition, we show how attributions enable lower-level behavior analysis in EAI
policies for language and visual attributions.

理解多模态感知以支持具体人工智能是一个开放性问题，因为这些输入可能既包含高度互补的信息，也存在冗余的信息。我们提出了 MAEA 框架，以计算每个可微策略的模态全局归属，并展示了如何应用归属性分析 EAI 策略中的语言和视觉归属性进行低层行为分析。