Natural language understanding (NLU) is the task of semantic decoding of
human languages by machines. NLU models rely heavily on large training data to
ensure good performance. However, substantial languages and domains have very
few data resources and domain experts. It is necessary to overcome the data
scarcity challenge, when very few or even zero training samples are available.
In this thesis, we focus on developing cross-lingual and cross-domain methods
to tackle the low-resource issues. First, we propose to improve the model's
cross-lingual ability by focusing on the task-related keywords, enhancing the
model's robustness and regularizing the representations. We find that the
representations for low-resource languages can be easily and greatly improved
by focusing on just the keywords. Second, we present Order-Reduced Modeling
methods for the cross-lingual adaptation, and find that modeling partial word
orders instead of the whole sequence can improve the robustness of the model
against word order differences between languages and task knowledge transfer to
low-resource languages. Third, we propose to leverage different levels of
domain-related corpora and additional masking of data in the pre-training for
the cross-domain adaptation, and discover that more challenging pre-training
can better address the domain discrepancy issue in the task knowledge transfer.
Finally, we introduce a coarse-to-fine framework, Coach, and a cross-lingual
and cross-domain parsing framework, X2Parser. Coach decomposes the
representation learning process into a coarse-grained and a fine-grained
feature learning, and X2Parser simplifies the hierarchical task structures into
flattened ones. We observe that simplifying task structures makes the
representation learning more effective for low-resource languages and domains.

本论文旨在解决 NLU 模型在缺乏数据或知识资源时的挑战，提出了跨语言和跨域适应方法及面向低资源语言的关键词增强方法、序列颗粒化建模方法、多领域预训练方法和粗粒度到细粒度的表示学习框架。