Domain-specific language understanding requires integrating multiple pieces
of relevant contextual information. For example, we see both suicide and
depression-related behavior (multiple contexts) in the text ``I have a gun and
feel pretty bad about my life, and it wouldn't be the worst thing if I didn't
wake up tomorrow''. Domain specificity in self-attention architectures is
handled by fine-tuning on excerpts from relevant domain specific resources
(datasets and external knowledge - medical textbook chapters on mental health
diagnosis related to suicide and depression). We propose a modified
self-attention architecture Knowledge-infused Self Attention Transformer (KSAT)
that achieves the integration of multiple domain-specific contexts through the
use of external knowledge sources. KSAT introduces knowledge-guided biases in
dedicated self-attention layers for each knowledge source to accomplish this.
In addition, KSAT provides mechanics for controlling the trade-off between
learning from data and learning from knowledge. Our quantitative and
qualitative evaluations show that (1) the KSAT architecture provides novel
human-understandable ways to precisely measure and visualize the contributions
of the infused domain contexts, and (2) KSAT performs competitively with other
knowledge-infused baselines and significantly outperforms baselines that use
fine-tuning for domain-specific tasks.

通过使用来自相关领域的外部知识资源，在 KSAT 中引入了知识注入的自我关注层，从而实现了对多个领域特定上下文的集成。KSAT 提供了控制从数据中学习与从知识中学习之间的权衡的机制，并与其他知识注入基线相竞争，显着优于使用精调进行领域特定任务的基线。

KSAT: 知识注入的自注意力变形器 —— 整合多个领域特定语境

KSAT: Knowledge-infused Self Attention Transformer -- Integrating Multiple Domain-Specific Contexts

The accuracy of prosodic structure prediction is crucial to the naturalness
of synthesized speech in Mandarin text-to-speech system, but now is limited by
widely-used sequence-to-sequence framework and error accumulation from previous
word segmentation results. In this paper, we propose a span-based Mandarin
prosodic structure prediction model to obtain an optimal prosodic structure
tree, which can be converted to corresponding prosodic label sequence. Instead
of the prerequisite for word segmentation, rich linguistic features are
provided by Chinese character-level BERT and sent to encoder with
self-attention architecture. On top of this, span representation and label
scoring are used to describe all possible prosodic structure trees, of which
each tree has its corresponding score. To find the optimal tree with the
highest score for a given sentence, a bottom-up CKY-style algorithm is further
used. The proposed method can predict prosodic labels of different levels at
the same time and accomplish the process directly from Chinese characters in an
end-to-end manner. Experiment results on two real-world datasets demonstrate
the excellent performance of our span-based method over all
sequence-to-sequence baseline approaches.

本文提出了一种基于 BERT 和自注意力结构实现的基于跨度的汉语韵律结构预测模型，通过对所提供的语言学特征进行编码得到最优韵律结构树，再利用自下而上的 CKY 算法找到得分最高的韵律树，从而完成汉字转韵律标签的端到端预测，实验结果显示：该方法比所有基于序列到序列的基线方法都要优异。

一个基于字级别跨度的汉语韵律结构预测模型

A Character-level Span-based Model for Mandarin Prosodic Structure Prediction

The goal of this work is to recognise phrases and sentences being spoken by a
talking face, with or without the audio. Unlike previous works that have
focussed on recognising a limited number of words or phrases, we tackle lip
reading as an open-world problem - unconstrained natural language sentences,
and in the wild videos. Our key contributions are: (1) we compare two models
for lip reading, one using a CTC loss, and the other using a
sequence-to-sequence loss. Both models are built on top of the transformer
self-attention architecture; (2) we investigate to what extent lip reading is
complementary to audio speech recognition, especially when the audio signal is
noisy; (3) we introduce and publicly release a new dataset for audio-visual
speech recognition, LRS2-BBC, consisting of thousands of natural sentences from
British television. The models that we train surpass the performance of all
previous work on a lip reading benchmark dataset by a significant margin.

本论文旨在识别带有或不带有音频的说话者嘴唇所述的短语和句子，我们提出了使用自注意力机制的 CTC 和序列到序列两种模型进行唇语识别，并研究唇语识别在有噪音的情况下与音频识别的互补性，同时我们介绍并公开发布了英国电视上成千上万自然语言的新数据集 LRS2-BBC，我们建立的模型在实验中的表现均超过了以前的相关工作。