Accurate modeling of the vocal tract is necessary to construct articulatory
representations for interpretable speech processing and linguistics. However,
vocal tract modeling is challenging because many internal articulators are
occluded from external motion capture technologies. Real-time magnetic
resonance imaging (RT-MRI) allows measuring precise movements of internal
articulators during speech, but annotated datasets of MRI are limited in size
due to time-consuming and computationally expensive labeling methods. We first
present a deep labeling strategy for the RT-MRI video using a vision-only
segmentation approach. We then introduce a multimodal algorithm using audio to
improve segmentation of vocal articulators. Together, we set a new benchmark
for vocal tract modeling in MRI video segmentation and use this to release
labels for a 75-speaker RT-MRI dataset, increasing the amount of labeled public
RT-MRI data of the vocal tract by over a factor of 9. The code and dataset
labels can be found at \url{rishiraij.github.io/multimodal-mri-avatar/}.

利用实时磁共振成像技术进行声道建模时的挑战和解决方案，通过视觉分割和多模态算法提高 MR 图像中声道部分的分割准确性，并释放了一个包含 75 个说话人的 RT-MRI 数据集的标注，增加了 9 倍以上的公共 RT-MRI 声道数据数量。