We proposed Audio Difference Captioning (ADC) as a new extension task of audio captioning for describing the semantic differences between input pairs of similar but slightly different audio clips. The ADC solves the problem that conventional audio captioning sometimes generates similar captions for similar audio clips, failing to describe the difference in content. We also propose a cross-attention-concentrated transformer encoder to extract differences by comparing a pair of audio clips and a similarity-discrepancy disentanglement to emphasize the difference in the latent space. To evaluate the proposed methods, we built an AudioDiffCaps dataset consisting of pairs of similar but slightly different audio clips with human-annotated descriptions of their differences. The experiment with the AudioDiffCaps dataset showed that the proposed methods solve the ADC task effectively and improve the attention weights to extract the difference by visualizing them in the transformer encoder.

对类似但略有不同的音频剪辑进行语义差异描述的音频差异字幕（ADC）是一项新的扩展任务，通过比较一对音频剪辑和强调潜在空间中的差异以提取差异的交叉注意力集中型变压器编码器来解决常规音频字幕生成类似标题的问题。

利用相似性差异解耦的音频差异字幕生成