We present a masked vision-language transformer (MVLT) for fashion-specific
multi-modal representation. Technically, we simply utilize vision transformer
architecture for replacing the BERT in the pre-training model, making MVLT the
first end-to-end framework for the fashion domain. Besides, we designed masked
image reconstruction (MIR) for a fine-grained understanding of fashion. MVLT is
an extensible and convenient architecture that admits raw multi-modal inputs
without extra pre-processing models (e.g., ResNet), implicitly modeling the
vision-language alignments. More importantly, MVLT can easily generalize to
various matching and generative tasks. Experimental results show obvious
improvements in retrieval (rank@5: 17%) and recognition (accuracy: 3%) tasks
over the Fashion-Gen 2018 winner Kaleido-BERT. Code is made available at
this https URL.

设计一种面向时尚领域的多模式表达模型，使用视觉转换器架构代替预训练模型 BERT，实现端到端框架，并使用遮蔽图像重构实现对时尚领域的细粒度理解。该模型没有使用额外的预处理模型（如 ResNet），能轻松推广到各种匹配和生成任务中，并得到了提取（rank@5: 17%）和识别（准确度: 3%）任务结果的明显改进。

时尚领域的遮蔽视觉语言变压器

Masked Vision-Language Transformer in Fashion

Document-level relation extraction aims to extract relations among entities
within a document. Compared with its sentence-level counterpart, Document-level
relation extraction requires inference over multiple sentences to extract
complex relational triples. Previous research normally complete reasoning
through information propagation on the mention-level or entity-level
document-graphs, regardless of the correlations between the relationships. In
this paper, we propose a novel Document-level Relation Extraction model based
on a Masked Image Reconstruction network (DRE-MIR), which models inference as a
masked image reconstruction problem to capture the correlations between
relationships. Specifically, we first leverage an encoder module to get the
features of entities and construct the entity-pair matrix based on the
features. After that, we look on the entity-pair matrix as an image and then
randomly mask it and restore it through an inference module to capture the
correlations between the relationships. We evaluate our model on three public
document-level relation extraction datasets, i.e. DocRED, CDR, and GDA.
Experimental results demonstrate that our model achieves state-of-the-art
performance on these three datasets and has excellent robustness against the
noises during the inference process.

提出了一种基于 Masked Image Reconstruction 网络的文档级关系提取模型（DRE-MIR），通过模型推理来捕捉关系之间的相关性，并在三个公共数据集上取得了最先进的性能。