We present an approach to simultaneously perform semantic segmentation and
prepositional phrase attachment resolution for captioned images. Some
ambiguities in language cannot be resolved without simultaneously reasoning
about an associated image. If we consider the sentence "I shot an elephant in
my pajamas", looking at language alone (and not using common sense), it is
unclear if it is the person or the elephant wearing the pajamas or both. Our
approach produces a diverse set of plausible hypotheses for both semantic
segmentation and prepositional phrase attachment resolution that are then
jointly reranked to select the most consistent pair. We show that our semantic
segmentation and prepositional phrase attachment resolution modules have
complementary strengths, and that joint reasoning produces more accurate
results than any module operating in isolation. Multiple hypotheses are also
shown to be crucial to improved multiple-module reasoning. Our vision and
language approach significantly outperforms the Stanford Parser (De Marneffe et
al., 2006) by 17.91% (28.69% relative) and 12.83% (25.28% relative) in two
different experiments. We also make small improvements over DeepLab-CRF (Chen
et al., 2015).

本文针对语言中的歧义问题提出了一种基于图片和联合排序的语义分割和介词短语附着识别方法，采用多个假设的联合排序方法比语言模型单独操作更准确。