Speech-to-text (S2T) summarization is a time-saving technique for filtering
and keeping up with the broadcast news uploaded online on a daily basis. The
rise of large language models from deep learning with impressive text
generation capabilities has placed the research focus on summarization systems
that produce paraphrased compact versions of the document content, also known
as abstractive summaries. End-to-end (E2E) modelling of S2T abstractive
summarization is a promising approach that offers the possibility of generating
rich latent representations that leverage non-verbal and acoustic information,
as opposed to the use of only linguistic information from automatically
generated transcripts in cascade systems. However, the few literature on E2E
modelling of this task fails on exploring different domains, namely broadcast
news, which is challenging domain where large and diversified volumes of data
are presented to the user every day. We model S2T summarization both with a
cascade and an E2E system for a corpus of broadcast news in French. Our novel
E2E model leverages external data by resorting to transfer learning from a
pre-trained T2T summarizer. Experiments show that both our cascade and E2E
abstractive summarizers are stronger than an extractive baseline. However, the
performance of the E2E model still lies behind the cascade one, which is object
of an extensive analysis that includes future directions to close that gap.

将语音文字（S2T）摘要分为级联和端到端两种模型，并使用预训练的 T2T 摘要提取器进行迁移学习，进而将其应用于在线上传的广播新闻中。结果表明，端到端模型要略逊于级联模型，但都优于提取式基线模型。

走向端到端的语音转文本摘要

Towards End-to-end Speech-to-text Summarization

Multilingual speech recognition has drawn significant attention as an
effective way to compensate data scarcity for low-resource languages.
End-to-end (e2e) modelling is preferred over conventional hybrid systems,
mainly because of no lexicon requirement. However, hybrid DNN-HMMs still
outperform e2e models in limited data scenarios. Furthermore, the problem of
manual lexicon creation has been alleviated by publicly available trained
models of grapheme-to-phoneme (G2P) and text to IPA transliteration for a lot
of languages. In this paper, a novel approach of hybrid DNN-HMM acoustic models
fusion is proposed in a multilingual setup for the low-resource languages.
Posterior distributions from different monolingual acoustic models, against a
target language speech signal, are fused together. A separate regression neural
network is trained for each source-target language pair to transform posteriors
from source acoustic model to the target language. These networks require very
limited data as compared to the ASR training. Posterior fusion yields a
relative gain of 14.65% and 6.5% when compared with multilingual and
monolingual baselines respectively. Cross-lingual model fusion shows that the
comparable results can be achieved without using posteriors from the language
dependent ASR.

在多语种语音识别中，为了弥补低资源语言的数据匮乏，人们广泛使用多功能语音识别。 本文提出了一种新颖的混合 DNN-HMM 声学模型融合方法，实现了多种低资源语言的跨语音识别，且相对于多功能和单语基线，后验融合得到了 14.65％和 6.5％的相对收益。