The challenge posed by multimodal named entity recognition (MNER) is mainly
two-fold: (1) bridging the semantic gap between text and image and (2) matching
the entity with its associated object in image. Existing methods fail to
capture the implicit entity-object relations, due to the lack of corresponding
annotation. In this paper, we propose a bidirectional generative alignment
method named BGA-MNER to tackle these issues. Our BGA-MNER consists of
\texttt{image2text} and \texttt{text2image} generation with respect to
entity-salient content in two modalities. It jointly optimizes the
bidirectional reconstruction objectives, leading to aligning the implicit
entity-object relations under such direct and powerful constraints.
Furthermore, image-text pairs usually contain unmatched components which are
noisy for generation. A stage-refined context sampler is proposed to extract
the matched cross-modal content for generation. Extensive experiments on two
benchmarks demonstrate that our method achieves state-of-the-art performance
without image input during inference.

该研究提出了一种名为 BGA-MNER 的双向生成对齐方法，用于解决多模态命名实体识别中的语义鸿沟、实体与图像中相关对象之间的匹配以及隐式实体 - 对象关系等问题。实验证明，该方法在推断过程中无需图像输入，取得了最先进的性能。