Multimodal Named Entity Recognition (MNER) is a crucial task for information
extraction from social media platforms such as Twitter. Most current methods
rely on attention weights to extract information from both text and images but
are often unreliable and lack interpretability. To address this problem, we
propose incorporating uncertainty estimation into the MNER task, producing
trustworthy predictions. Our proposed algorithm models the distribution of each
modality as a Normal-inverse Gamma distribution, and fuses them into a unified
distribution with an evidential fusion mechanism, enabling hierarchical
characterization of uncertainties and promotion of prediction accuracy and
trustworthiness. Additionally, we explore the potential of pre-trained large
foundation models in MNER and propose an efficient fusion approach that
leverages their robust feature representations. Experiments on two datasets
demonstrate that our proposed method outperforms the baselines and achieves new
state-of-the-art performance.

提出一种结合不确定性评估的多模态命名实体识别框架，该算法模拟每种形式的分布并将它们融合成统一分布，以促进预测准确性和可信度。同时提出了一种利用大型预训练基础模型的高效融合方法，实验表明该方法优于基准并实现了新的最先进性能。