Discrete audio tokens have recently gained considerable attention for their
potential to connect audio and language processing, enabling the creation of
modern multimodal large language models. Ideal audio tokens must effectively
preserve phonetic and semantic content along with paralinguistic information,
speaker identity, and other details. While several types of audio tokens have
been recently proposed, identifying the optimal tokenizer for various tasks is
challenging due to the inconsistent evaluation settings in existing studies. To
address this gap, we release the Discrete Audio and Speech Benchmark (DASB), a
comprehensive leaderboard for benchmarking discrete audio tokens across a wide
range of discriminative tasks, including speech recognition, speaker
identification and verification, emotion recognition, keyword spotting, and
intent classification, as well as generative tasks such as speech enhancement,
separation, and text-to-speech. Our results show that, on average, semantic
tokens outperform compression tokens across most discriminative and generative
tasks. However, the performance gap between semantic tokens and standard
continuous representations remains substantial, highlighting the need for
further research in this field.

通过发布离散音频和语音基准 (DASB)，我们研究了离散音频标记符在各种任务中的性能表现，并发现语义标记比压缩标记在大多数任务中表现更好，但语义标记和标准连续表示之间的性能差距仍然存在。

DASB -- 离散音频和语音基准

DASB -- Discrete Audio and Speech Benchmark

Discrete audio tokens have recently gained attention for their potential to
bridge the gap between audio and language processing. Ideal audio tokens must
preserve content, paralinguistic elements, speaker identity, and many other
audio details. Current audio tokenization methods fall into two categories:
Semantic tokens, acquired through quantization of Self-Supervised Learning
(SSL) models, and Neural compression-based tokens (codecs). Although previous
studies have benchmarked codec models to identify optimal configurations, the
ideal setup for quantizing pretrained SSL models remains unclear. This paper
explores the optimal configuration of semantic tokens across discriminative and
generative tasks. We propose a scalable solution to train a universal vocoder
across multiple SSL layers. Furthermore, an attention mechanism is employed to
identify task-specific influential layers, enhancing the adaptability and
performance of semantic tokens in diverse audio applications.

这篇论文探讨了语义标记的最佳配置，提出了训练通用声码器的可扩展解决方案，并使用注意机制来提高语义标记在多种音频应用中的适应性和性能。