Information in industry, research, and the public sector is widely stored as
rendered documents (e.g., PDF files, scans). Hence, to enable downstream tasks,
systems are needed that map rendered documents onto a structured hierarchical
format. However, existing systems for this task are limited by heuristics and
are not end-to-end trainable. In this work, we introduce the Document Structure
Generator (DSG), a novel system for document parsing that is fully end-to-end
trainable. DSG combines a deep neural network for parsing (i) entities in
documents (e.g., figures, text blocks, headers, etc.) and (ii) relations that
capture the sequence and nested structure between entities. Unlike existing
systems that rely on heuristics, our DSG is trained end-to-end, making it
effective and flexible for real-world applications. We further contribute a
new, large-scale dataset called E-Periodica comprising real-world magazines
with complex document structures for evaluation. Our results demonstrate that
our DSG outperforms commercial OCR tools and, on top of that, achieves
state-of-the-art performance. To the best of our knowledge, our DSG system is
the first end-to-end trainable system for hierarchical document parsing.

在这项研究中，我们介绍了一种名为 DSG 的文档解析系统，它是一种全面可训练的端到端系统，用于将渲染文档映射到结构化的分层格式。我们通过训练 DSG 系统，使其在实际应用中具有高效和灵活的功能，并在评估中证明 DSG 优于商业 OCR 工具并且达到了最先进的性能水平。据我们所知，我们的 DSG 系统是第一个进行层次化文档解析的全面可训练系统。

DSG：一种端到端的文档结构生成器

DSG: An End-to-End Document Structure Generator

Compatibility between items, such as clothes and shoes, is a major factor
among customer's purchasing decisions. However, learning "compatibility" is
challenging due to (1) broader notions of compatibility than those of
similarity, (2) the asymmetric nature of compatibility, and (3) only a small
set of compatible and incompatible items are observed. We propose an end-to-end
trainable system to embed each item into a latent vector and project a query
item into K compatible prototypes in the same space. These prototypes reflect
the broad notions of compatibility. We refer to both the embedding and
prototypes as "Compatibility Family". In our learned space, we introduce a
novel Projected Compatibility Distance (PCD) function which is differentiable
and ensures diversity by aiming for at least one prototype to be close to a
compatible item, whereas none of the prototypes are close to an incompatible
item. We evaluate our system on a toy dataset, two Amazon product datasets, and
Polyvore outfit dataset. Our method consistently achieves state-of-the-art
performance. Finally, we show that we can visualize the candidate compatible
prototypes using a Metric-regularized Conditional Generative Adversarial
Network (MrCGAN), where the input is a projected prototype and the output is a
generated image of a compatible item. We ask human evaluators to judge the
relative compatibility between our generated images and images generated by
CGANs conditioned directly on query items. Our generated images are
significantly preferred, with roughly twice the number of votes as others.

通过将每个物品嵌入到潜在向量中，并将查询项目投影到 K 个兼容原型中，在我们学习的空间中，我们介绍了一种新颖的投影兼容性距离（PCD）函数，该函数是可微分的并确保多样性。我们的系统在玩具数据集，两个亚马逊产品数据集和 Polyvore 服装数据集上得到了实验验证，表现始终处于最前沿。最后，我们展示了我们可以使用度量正则化的条件生成对抗网络（MrCGAN）可视化候选兼容的原型。