Question Answering (QA) is a task that entails reasoning over natural
language contexts, and many relevant works augment language models (LMs) with
graph neural networks (GNNs) to encode the Knowledge Graph (KG) information.
However, most existing GNN-based modules for QA do not take advantage of rich
relational information of KGs and depend on limited information interaction
between the LM and the KG. To address these issues, we propose Question
Answering Transformer (QAT), which is designed to jointly reason over language
and graphs with respect to entity relations in a unified manner. Specifically,
QAT constructs Meta-Path tokens, which learn relation-centric embeddings based
on diverse structural and semantic relations. Then, our Relation-Aware
Self-Attention module comprehensively integrates different modalities via the
Cross-Modal Relative Position Bias, which guides information exchange between
relevant entities of different modalities. We validate the effectiveness of QAT
on commonsense question answering datasets like CommonsenseQA and OpenBookQA,
and on a medical question answering dataset, MedQA-USMLE. On all the datasets,
our method achieves state-of-the-art performance. Our code is available at
this http URL.

提出了一种可以利用 KG 丰富的关系信息和 LM 之间有限的交互方式，联合处理自然语言和图表信息的方法，并通过 CommonsenseQA 和 OpenBookQA 等数据集的验证，在所有数据集上都取得了最先进的性能。

关系感知语言 - 图变换器用于问答

Relation-aware Language-Graph Transformer for Question Answering

Relational structures such as schema linking and schema encoding have been
validated as a key component to qualitatively translating natural language into
SQL queries. However, introducing these structural relations comes with prices:
they often result in a specialized model structure, which largely prohibits
using large pretrained models in text-to-SQL. To address this problem, we
propose RASAT: a Transformer seq2seq architecture augmented with relation-aware
self-attention that could leverage a variety of relational structures while
inheriting the pretrained parameters from the T5 model effectively. Our model
can incorporate almost all types of existing relations in the literature, and
in addition, we propose introducing co-reference relations for the multi-turn
scenario. Experimental results on three widely used text-to-SQL datasets,
covering both single-turn and multi-turn scenarios, have shown that RASAT could
achieve state-of-the-art results across all three benchmarks (75.5% EX on
Spider, 52.6% IEX on SParC, and 37.4% IEX on CoSQL).

使用关系结构提高 Transformer seq2seq 模型在文本转 SQL 中的效果，实验结果表明在多个数据集中取得了最优结果。

RASAT：将关系结构集成到预训练 Seq2Seq 模型中，用于文本到 SQL 的转换

RASAT: Integrating Relational Structures into Pretrained Seq2Seq Model for Text-to-SQL

When translating natural language questions into SQL queries to answer
questions from a database, we would like our methods to generalize to domains
and database schemas outside of the training set. To handle complex questions
and database schemas with a neural encoder-decoder paradigm, it is critical to
properly encode the schema as part of the input with the question. In this
paper, we use relation-aware self-attention within the encoder so that it can
reason about how the tables and columns in the provided schema relate to each
other and use this information in interpreting the question. We achieve
significant gains on the recently-released Spider dataset with 42.94% exact
match accuracy, compared to the 18.96% reported in published work.

通过使用自我关联，让编码器理解表格和列之间的关系，以及将该信息使用在问题的解释过程中，使得神经编码器 - 解码器系统能够在 SQL 查询的实现上具有较好的泛化性，从而在 Spider 数据集上获得 42.94% 的精确匹配准确率。