It is an open challenge to obtain high quality training data, especially
captions, for text-to-audio models. Although prior methods have leveraged
\textit{text-only language models} to augment and improve captions, such
methods have limitations related to scale and coherence between audio and
captions. In this work, we propose an audio captioning pipeline that uses an
\textit{audio language model} to synthesize accurate and diverse captions for
audio at scale. We leverage this pipeline to produce a dataset of synthetic
captions for AudioSet, named \texttt{AF-AudioSet}, and then evaluate the
benefit of pre-training text-to-audio models on these synthetic captions.
Through systematic evaluations on AudioCaps and MusicCaps, we find leveraging
our pipeline and synthetic captions leads to significant improvements on audio
generation quality, achieving a new \textit{state-of-the-art}.

通过语音标注管道，使用音频语言模型合成准确且多样的音频标注，提供了合成标注数据集以及训练基于合成标注的文本转音频模型，取得了新的最先进的结果。

用合成标题改进文本到音频模型

Improving Text-To-Audio Models with Synthetic Captions

With the development of AI-Generated Content (AIGC), text-to-audio models are
gaining widespread attention. However, it is challenging for these models to
generate audio aligned with human preference due to the inherent information
density of natural language and limited model understanding ability. To
alleviate this issue, we formulate the BATON, a framework designed to enhance
the alignment between generated audio and text prompt using human preference
feedback. Our BATON comprises three key stages: Firstly, we curated a dataset
containing both prompts and the corresponding generated audio, which was then
annotated based on human feedback. Secondly, we introduced a reward model using
the constructed dataset, which can mimic human preference by assigning rewards
to input text-audio pairs. Finally, we employed the reward model to fine-tune
an off-the-shelf text-to-audio model. The experiment results demonstrate that
our BATON can significantly improve the generation quality of the original
text-to-audio models, concerning audio integrity, temporal relationship, and
alignment with human preference.

使用人类偏好反馈，通过 BATON 框架增强生成音频与文本提示之间的对齐，从而显著提高原始文本到音频模型的生成质量。