In the FAME! project, we aim to develop an automatic speech recognition (ASR)
system for Frisian-Dutch code-switching (CS) speech extracted from the archives
of a local broadcaster with the ultimate goal of building a spoken document
retrieval system. Unlike Dutch, Frisian is a low-resourced language with a very
limited amount of manually annotated speech data. In this paper, we describe
several automatic annotation approaches to enable using of a large amount of
raw bilingual broadcast data for acoustic model training in a semi-supervised
setting. Previously, it has been shown that the best-performing ASR system is
obtained by two-stage multilingual deep neural network (DNN) training using 11
hours of manually annotated CS speech (reference) data together with speech
data from other high-resourced languages. We compare the quality of
transcriptions provided by this bilingual ASR system with several other
approaches that use a language recognition system for assigning language labels
to raw speech segments at the front-end and using monolingual ASR resources for
transcription. We further investigate automatic annotation of the speakers
appearing in the raw broadcast data by first labeling with (pseudo) speaker
tags using a speaker diarization system and then linking to the known speakers
appearing in the reference data using a speaker recognition system. These
speaker labels are essential for speaker-adaptive training in the proposed
setting. We train acoustic models using the manually and automatically
annotated data and run recognition experiments on the development and test data
of the FAME! speech corpus to quantify the quality of the automatic
annotations. The ASR and CS detection results demonstrate the potential of
using automatic language and speaker tagging in semi-supervised bilingual
acoustic model training.

本文主要介绍了对 Frisian—Dutch code-switching 语音进行自动化标注和学习的几种方法，并探索了扩充语音数据的方法以提高类似语音识别模型的效果的研究。