Recent studies have advocated for fully open foundation models to promote
transparency and open science. As an initial step, the Open Whisper-style
Speech Model (OWSM) reproduced OpenAI's Whisper using publicly available data
and open-source toolkits. With the aim of reproducing Whisper, the previous
OWSM v1 through v3 models were still based on Transformer, which might lead to
inferior performance compared to other state-of-the-art speech encoders. In
this work, we aim to improve the performance and efficiency of OWSM without
extra training data. We present E-Branchformer based OWSM v3.1 models at two
scales, i.e., 100M and 1B. The 1B model is the largest E-Branchformer based
speech model that has been made publicly available. It outperforms the previous
OWSM v3 in a vast majority of evaluation benchmarks, while demonstrating up to
25% faster inference speed. We publicly release the data preparation scripts,
pre-trained models and training logs.

利用 E-Branchformer 提高 OWSM v3.1 模型的性能和效率，尤其在推理速度上具有 25% 的提升，并推出具有 1B 规模的最大 E-Branchformer 语音模型。

OWSM v3.1：基于 E-Branchformer 的更好更快的开放式耳语口语模型

OWSM v3.1: Better and Faster Open Whisper-Style Speech Models based on  E-Branchformer

Conformer, a convolution-augmented Transformer variant, has become the de
facto encoder architecture for speech processing due to its superior
performance in various tasks, including automatic speech recognition (ASR),
speech translation (ST) and spoken language understanding (SLU). Recently, a
new encoder called E-Branchformer has outperformed Conformer in the LibriSpeech
ASR benchmark, making it promising for more general speech applications. This
work compares E-Branchformer and Conformer through extensive experiments using
different types of end-to-end sequence-to-sequence models. Results demonstrate
that E-Branchformer achieves comparable or better performance than Conformer in
almost all evaluation sets across 15 ASR, 2 ST, and 3 SLU benchmarks, while
being more stable during training. We will release our training configurations
and pre-trained models for reproducibility, which can benefit the speech
community.

本文比较 E-Branchformer 和 Conformer 两种编码器在语音处理方面的性能，并表明 E-Branchformer 在大多数评估数据集上表现比 Conformer 更好，并且在训练期间更加稳定。