We propose an automated image selection system to assist photo editors in
selecting suitable images for news articles. The system fuses multiple textual
sources extracted from news articles and accepts multilingual inputs. It is
equipped with char-level word embeddings to help both modeling morphologically
rich languages, e.g. German, and transferring knowledge across nearby
languages. The text encoder adopts a hierarchical self-attention mechanism to
attend more to both keywords within a piece of text and informative components
of a news article. We extensively experiment with our system on a large-scale
text-image database containing multimodal multilingual news articles collected
from Swiss local news media websites. The system is compared with multiple
baselines with ablation studies and is shown to beat existing text-image
retrieval methods in a weakly-supervised learning setting. Besides, we also
offer insights on the advantage of using multiple textual sources and
multilingual data.

本文提出了一个自动化图像选择系统，用于协助编辑在新闻文章中选择合适的图片。该系统融合了从新闻文章中提取的多个文本来源，并接受多语言输入。它配备有字符级词嵌入，以帮助对形态丰富的语言建模，并在靠近的语言之间转移知识。文本编码器采用分层自注意机制，更多地关注文本中的关键字和新闻文章的信息组成部分。我们在瑞士本地新闻媒体网站收集的多模态多语言新闻文章的大规模文本 - 图像数据库上进行了广泛的实验。在弱监督学习设置中，将该系统与多个基线方法进行比较，并显示超过了现有的文本 - 图像检索方法。除此之外，我们还提供了使用多个文本来源和多语言数据的优势见解。