Current Conversational AI systems employ different machine learning
pipelines, as well as external knowledge sources and business logic to predict
the next action. Maintaining various components in dialogue managers' pipeline
adds complexity in expansion and updates, increases processing time, and causes
additive noise through the pipeline that can lead to incorrect next action
prediction. This paper investigates graph integration into language
transformers to improve understanding the relationships between humans'
utterances, previous, and next actions without the dependency on external
sources or components. Experimental analyses on real calls indicate that the
proposed Graph Integrated Language Transformer models can achieve higher
performance compared to other production level conversational AI systems in
driving interactive calls with human users in real-world settings.

本文通过将图形集成到语言转换器中，改善了理解人类话语、先前和下一个动作之间关系的能力，而无需依赖外部来源或组件。实验分析表明，所提出的图形集成语言转换器模型在实际调用中可以达到比其他生产级对话式人工智能系统更高的性能。

图结合的语言转换器在复杂电话呼叫中的下一步动作预测

Graph Integrated Language Transformers for Next Action Prediction in  Complex Phone Calls

The activations of language transformers like GPT-2 have been shown to
linearly map onto brain activity during speech comprehension. However, the
nature of these activations remains largely unknown and presumably conflate
distinct linguistic classes. Here, we propose a taxonomy to factorize the
high-dimensional activations of language models into four combinatorial
classes: lexical, compositional, syntactic, and semantic representations. We
then introduce a statistical method to decompose, through the lens of GPT-2's
activations, the brain activity of 345 subjects recorded with functional
magnetic resonance imaging (fMRI) during the listening of ~4.6 hours of
narrated text. The results highlight two findings. First, compositional
representations recruit a more widespread cortical network than lexical ones,
and encompass the bilateral temporal, parietal and prefrontal cortices. Second,
contrary to previous claims, syntax and semantics are not associated with
separated modules, but, instead, appear to share a common and distributed
neural substrate. Overall, this study introduces a versatile framework to
isolate, in the brain activity, the distributed representations of linguistic
constructs.

通过对 GPT-2 的激活进行统计的方法，将语言模型的高维度激活分解成四个组合类别：词汇、组合、句法和语义表示，并发现组合表示比词汇表示更广泛地招募了大脑皮层网络，并且句法和语义共享一个分布式神经基础。该研究提出了一个多功能的框架来分离神经活动中的语言建构分布式表示。