Natural language understanding (NLU) is an essential branch of natural
language processing, which relies on representations generated by pre-trained
language models (PLMs). However, PLMs primarily focus on acquiring
lexico-semantic information, while they may be unable to adequately handle the
meaning of constructions. To address this issue, we introduce construction
grammar (CxG), which highlights the pairings of form and meaning, to enrich
language representation. We adopt usage-based construction grammar as the basis
of our work, which is highly compatible with statistical models such as PLMs.
Then a HyCxG framework is proposed to enhance language representation through a
three-stage solution. First, all constructions are extracted from sentences via
a slot-constraints approach. As constructions can overlap with each other,
bringing redundancy and imbalance, we formulate the conditional max coverage
problem for selecting the discriminative constructions. Finally, we propose a
relational hypergraph attention network to acquire representation from
constructional information by capturing high-order word interactions among
constructions. Extensive experiments demonstrate the superiority of the
proposed model on a variety of NLU tasks.

本文介绍了一种使用构造语法概念来丰富语言表示的方法，该方法由一种基于使用的构造语法框架支持，通过提取具有辨别性的构造，然后利用关系超图注意网络获取构造信息的表示来增强语言表示，实验结果表明该模型在各种自然语言理解任务方面具有明显优势。