Echocardiography (ECHO) video is widely used for cardiac examination. In
clinical, this procedure heavily relies on operator experience, which needs
years of training and maybe the assistance of deep learning-based systems for
enhanced accuracy and efficiency. However, it is challenging since acquiring
sufficient customized data (e.g., abnormal cases) for novice training and deep
model development is clinically unrealistic. Hence, controllable ECHO video
synthesis is highly desirable. In this paper, we propose a novel
diffusion-based framework named HeartBeat towards controllable and
high-fidelity ECHO video synthesis. Our highlight is three-fold. First,
HeartBeat serves as a unified framework that enables perceiving multimodal
conditions simultaneously to guide controllable generation. Second, we
factorize the multimodal conditions into local and global ones, with two
insertion strategies separately provided fine- and coarse-grained controls in a
composable and flexible manner. In this way, users can synthesize ECHO videos
that conform to their mental imagery by combining multimodal control signals.
Third, we propose to decouple the visual concepts and temporal dynamics
learning using a two-stage training scheme for simplifying the model training.
One more interesting thing is that HeartBeat can easily generalize to
mask-guided cardiac MRI synthesis in a few shots, showcasing its scalability to
broader applications. Extensive experiments on two public datasets show the
efficacy of the proposed HeartBeat.

该研究提出了一种名为 HeartBeat 的基于扩散的框架，用于可控和高保真度的 ECHO（超声心动图）视频合成，通过同时感知多模态条件，将这些条件分解为局部和全局条件，并使用两阶段训练方案解耦视觉概念和时间动态学习，以简化模型训练，该框架可扩展到其他广泛应用，如基于遮罩引导的心脏 MRI 合成。在两个公共数据集上进行的广泛实验证明了 HeartBeat 的有效性。