Compositional generalization, the ability of intelligent models to
extrapolate understanding of components to novel compositions, is a fundamental
yet challenging facet in AI research, especially within multimodal
environments. In this work, we address this challenge by exploiting the
syntactic structure of language to boost compositional generalization. This
paper elevates the importance of syntactic grounding, particularly through
attention masking techniques derived from text input parsing. We introduce and
evaluate the merits of using syntactic information in the multimodal grounding
problem. Our results on grounded compositional generalization underscore the
positive impact of dependency parsing across diverse tasks when utilized with
Weight Sharing across the Transformer encoder. The results push the
state-of-the-art in multimodal grounding and parameter-efficient modeling and
provide insights for future research.

利用句法结构的注意力屏蔽技术从根本上提高了组合概括的重要性，特别是在多模式环境中的语义根基问题。通过在 Transformer 编码器中的权重共享，依赖解析在不同任务中推动了基于语法的组合概括的最新研究。这些结果提高了多模式语义根基和参数高效建模的技术水平，并为未来的研究提供了深入洞察。