We present a prompt-engineering-based text-augmentation approach applied to a
language-queried audio source separation (LASS) task. To enhance the
performance of LASS, the proposed approach utilizes large language models
(LLMs) to generate multiple captions corresponding to each sentence of the
training dataset. To this end, we first perform experiments to identify the
most effective prompts for caption augmentation with a smaller number of
captions. A LASS model trained with these augmented captions demonstrates
improved performance on the DCASE 2024 Task 9 validation set compared to that
trained without augmentation. This study highlights the effectiveness of
LLM-based caption augmentation in advancing language-queried audio source
separation.

我们提出了一种基于提示工程的文本增强方法，应用于语言查询音频源分离（LASS）任务。该方法利用大型语言模型生成与训练数据集每个句子对应的多个字幕来增强 LASS 的性能。实验结果表明，使用这些增强的字幕训练的 LASS 模型在 DCASE 2024 任务 9 验证集上表现出了比没有增强的模型更好的性能。该研究突出了基于语言模型的字幕增强在推动语言查询音频源分离方面的有效性。

基于大型语言模型的字幕增强技术提升的语言查询音频源分离性能：DCASE Challenge 2024 任务 9

Performance Improvement of Language-Queried Audio Source Separation  Based on Caption Augmentation From Large Language Models for DCASE Challenge  2024 Task 9

Although contextualized automatic speech recognition (ASR) systems are
commonly used to improve the recognition of uncommon words, their effectiveness
is hindered by the inherent limitations of speech-text data availability. To
address this challenge, our study proposes to leverage extensive text-only
datasets and contextualize pre-trained ASR models using a straightforward
text-augmentation (TA) technique, all while keeping computational costs
minimal. In particular, to contextualize a pre-trained CIF-based ASR, we
construct a codebook using limited speech-text data. By utilizing a simple
codebook lookup process, we convert available text-only data into latent text
embeddings. These embeddings then enhance the inputs for the contextualized
ASR. Our experiments on diverse Mandarin test sets demonstrate that our TA
approach significantly boosts recognition performance. The top-performing
system shows relative CER improvements of up to 30% on rare words and 15%
across all words in general.

使用简单的文本增强技术借助大量纯文本数据集来构建编码簿，可以提高预训练的 ASR 模型的上下文信息，从而显著提升识别性能。