Audio captioning aims to automatically generate a natural language
description of an audio clip. Most captioning models follow an encoder-decoder
architecture, where the decoder predicts words based on the audio features
extracted by the encoder. Convolutional neural networks (CNNs) and recurrent
neural networks (RNNs) are often used as the audio encoder. However, CNNs can
be limited in modelling temporal relationships among the time frames in an
audio signal, while RNNs can be limited in modelling the long-range
dependencies among the time frames. In this paper, we propose an Audio
Captioning Transformer (ACT), which is a full Transformer network based on an
encoder-decoder architecture and is totally convolution-free. The proposed
method has a better ability to model the global information within an audio
signal as well as capture temporal relationships between audio events. We
evaluate our model on AudioCaps, which is the largest audio captioning dataset
publicly available. Our model shows competitive performance compared to other
state-of-the-art approaches.

该论文提出了一种全转换器的音频字幕生成器，该生成器能更好地模拟音频信号中的全局信息以及音频事件之间的时间关系，并且在最大的音频字幕数据集上显示出与其他最先进方法相竞争的性能。