Automatic pronunciation assessment is a major component of a
computer-assisted pronunciation training system. To provide in-depth feedback,
scoring pronunciation at various levels of granularity such as phoneme, word,
and utterance, with diverse aspects such as accuracy, fluency, and
completeness, is essential. However, existing multi-aspect multi-granularity
methods simultaneously predict all aspects at all granularity levels;
therefore, they have difficulty in capturing the linguistic hierarchy of
phoneme, word, and utterance. This limitation further leads to neglecting
intimate cross-aspect relations at the same linguistic unit. In this paper, we
propose a Hierarchical Pronunciation Assessment with Multi-aspect Attention
(HiPAMA) model, which hierarchically represents the granularity levels to
directly capture their linguistic structures and introduces multi-aspect
attention that reflects associations across aspects at the same level to create
more connotative representations. By obtaining relational information from both
the granularity- and aspect-side, HiPAMA can take full advantage of multi-task
learning. Remarkable improvements in the experimental results on the
speachocean762 datasets demonstrate the robustness of HiPAMA, particularly in
the difficult-to-assess aspects.

本研究提出了一种基于多方面注意力和分层表示的自动发音评估模型（ HiPAMA ），可在不同层次（如音素、单词和句子）上评估发音的多个方面（ 如准确性、流利度和完整性），从而实现更全面的反馈评估