End-to-end speech summarization (E2E SSum) directly summarizes input speech
into easy-to-read short sentences with a single model. This approach is
promising because it, in contrast to the conventional cascade approach, can
utilize full acoustical information and mitigate to the propagation of
transcription errors. However, due to the high cost of collecting
speech-summary pairs, an E2E SSum model tends to suffer from training data
scarcity and output unnatural sentences. To overcome this drawback, we propose
for the first time to integrate a pre-trained language model (LM), which is
highly capable of generating natural sentences, into the E2E SSum decoder via
transfer learning. In addition, to reduce the gap between the independently
pre-trained encoder and decoder, we also propose to transfer the baseline E2E
SSum encoder instead of the commonly used automatic speech recognition encoder.
Experimental results show that the proposed model outperforms baseline and data
augmented models.

本文提出在端对端语音摘要（E2E SSum）模型中集成预训练语言模型来解决数据稀缺问题，并通过迁移学习来减少编解码器之间的差距。实验证明，该模型表现优于基线和数据增强模型。