Benchmarking plays a pivotal role in assessing and enhancing the performance
of compact deep learning models designed for execution on resource-constrained
devices, such as microcontrollers. Our study introduces a novel, entirely
artificially generated benchmarking dataset tailored for speech recognition,
representing a core challenge in the field of tiny deep learning. SpokeN-100
consists of spoken numbers from 0 to 99 spoken by 32 different speakers in four
different languages, namely English, Mandarin, German and French, resulting in
12,800 audio samples. We determine auditory features and use UMAP (Uniform
Manifold Approximation and Projection for Dimension Reduction) as a
dimensionality reduction method to show the diversity and richness of the
dataset. To highlight the use case of the dataset, we introduce two benchmark
tasks: given an audio sample, classify (i) the used language and/or (ii) the
spoken number. We optimized state-of-the-art deep neural networks and performed
an evolutionary neural architecture search to find tiny architectures optimized
for the 32-bit ARM Cortex-M4 nRF52840 microcontroller. Our results represent
the first benchmark data achieved for SpokeN-100.

我们引入了一个新颖的、完全人工生成的适用于资源受限设备上执行的紧凑深度学习模型的性能评估和优化的基准数据集，该数据集特定用于语音识别，代表了微型深度学习领域中的一个核心挑战。SpokeN-100 数据集由 32 位不同说话者以四种不同语言（英语、普通话、德语和法语）发音的从 0 到 99 的数字组成，共包含 12,800 个音频样本。我们确定了听觉特征，并使用 UMAP 作为降维方法，展示了数据集的多样性和丰富性。为了突出数据集的应用案例，我们引入了两个基准任务：给定一个音频样本，分类（i）所使用的语言和 / 或（ii）所说的数字。我们对最先进的深度神经网络进行了优化，并进行了进化神经架构搜索，以找到适用于 32 位 ARM Cortex-M4 nRF52840 微控制器的紧凑架构。我们的结果是 SpokeN-100 数据集所达到的第一个基准数据。