This thesis investigates how natural language understanding and generation
with transformer models can benefit from grounding the models with knowledge
representations and addresses the following key research questions: (i) Can
knowledge of entities extend its benefits beyond entity-centric tasks, such as
entity linking? (ii) How can we faithfully and effectively extract such
structured knowledge from raw text, especially noisy web text? (iii) How do
other types of knowledge, beyond structured knowledge, contribute to improving
NLP tasks?
Studies in this thesis find that incorporating relevant and up-to-date
knowledge of entities benefits fake news detection, and entity-focused
code-switching significantly enhances zero-shot cross-lingual transfer on
entity-centric tasks. In terms of effective and faithful approaches to
extracting structured knowledge, it is observed that integrating negative
examples and training with entity planning significantly improves performance.
Additionally, it is established that other general forms of knowledge, such as
parametric and distilled knowledge, enhance multimodal and multilingual
knowledge-intensive tasks. This research shows the tangible benefits of diverse
knowledge integration and motivates further exploration in this direction.

通过将转换器模型与知识表示相关联，本论文研究了自然语言理解和生成的效果，并回答了以下关键研究问题：(i) 实体知识是否能够超越实体链接等实体为中心的任务，并带来更多的好处？(ii) 如何从原始文本中准确有效地提取这种结构化知识，尤其是嘈杂的网络文本？(iii) 除了结构化知识，其他类型的知识如何对改进自然语言处理任务有所贡献？研究表明，引入相关和最新的实体知识有助于虚假新闻检测，而面向实体的语言转换显著提高了零样本跨语言迁移的效果。在提取结构化知识方面，将负面样本整合和与实体规划一起训练显著提高了性能。此外，研究还发现，其他一般形式的知识，如参数化知识和精炼知识，有助于多模态和多语言知识密集型任务。这项研究展示了多元知识整合的实际好处，并鼓励在这个方向上进一步探索。

走向知识驱动的自然语言理解与生成

Towards Knowledge-Grounded Natural Language Understanding and Generation

There has been a steady need to precisely extract structured knowledge from
the web (i.e. HTML documents). Given a web page, extracting a structured object
along with various attributes of interest (e.g. price, publisher, author, and
genre for a book) can facilitate a variety of downstream applications such as
large-scale knowledge base construction, e-commerce product search, and
personalized recommendation. Considering each web page is rendered from an HTML
DOM tree, existing approaches formulate the problem as a DOM tree node tagging
task. However, they either rely on computationally expensive visual feature
engineering or are incapable of modeling the relationship among the tree nodes.
In this paper, we propose a novel transferable method, Simplified DOM Trees for
Attribute Extraction (SimpDOM), to tackle the problem by efficiently retrieving
useful context for each node by leveraging the tree structure. We study two
challenging experimental settings: (i) intra-vertical few-shot extraction, and
(ii) cross-vertical fewshot extraction with out-of-domain knowledge, to
evaluate our approach. Extensive experiments on the SWDE public dataset show
that SimpDOM outperforms the state-of-the-art (SOTA) method by 1.44% on the F1
score. We also find that utilizing knowledge from a different vertical
(cross-vertical extraction) is surprisingly useful and helps beat the SOTA by a
further 1.37%.

本文介绍了一种新方法 --SimpDOM，可以高效地从 HTML 文档中提取结构化对象的各种属性，为大规模知识库构建、电子商务产品搜索和个性化推荐等下游应用提供帮助，并通过在 SWDE 公共数据集上的广泛实验展示了 SimpDOM 的良好性能。