Magnetic resonance imaging~(MRI) have played a crucial role in brain disease
diagnosis, with which a range of computer-aided artificial intelligence methods
have been proposed. However, the early explorations usually focus on the
limited types of brain diseases in one study and train the model on the data in
a small scale, yielding the bottleneck of generalization. Towards a more
effective and scalable paradigm, we propose a hierarchical knowledge-enhanced
pre-training framework for the universal brain MRI diagnosis, termed as
UniBrain. Specifically, UniBrain leverages a large-scale dataset of 24,770
imaging-report pairs from routine diagnostics. Different from previous
pre-training techniques for the unitary vision or textual feature, or with the
brute-force alignment between vision and language information, we leverage the
unique characteristic of report information in different granularity to build a
hierarchical alignment mechanism, which strengthens the efficiency in feature
learning. Our UniBrain is validated on three real world datasets with severe
class imbalance and the public BraTS2019 dataset. It not only consistently
outperforms all state-of-the-art diagnostic methods by a large margin and
provides a superior grounding performance but also shows comparable performance
compared to expert radiologists on certain disease types.

磁共振成像（MRI）在大脑疾病诊断中起着至关重要的作用，本研究提出了一种分层知识增强的预训练框架 UniBrain，用于通用的脑 MRI 诊断，通过构建一种层次对齐机制，利用大规模数据集进行特征学习，取得了比所有最先进的诊断方法更好的性能且与专家放射科医生相媲美的表现。

UniBrain：基于层次化知识增强预训练的通用脑部 MRI 诊断

UniBrain: Universal Brain MRI Diagnosis with Hierarchical  Knowledge-enhanced Pre-training

Structural magnetic resonance imaging (sMRI) has shown great clinical value
and has been widely used in deep learning (DL) based computer-aided brain
disease diagnosis. Previous approaches focused on local shapes and textures in
sMRI that may be significant only within a particular domain. The learned
representations are likely to contain spurious information and have a poor
generalization ability in other diseases and datasets. To facilitate capturing
meaningful and robust features, it is necessary to first comprehensively
understand the intrinsic pattern of the brain that is not restricted within a
single data/task domain. Considering that the brain is a complex connectome of
interlinked neurons, the connectional properties in the brain have strong
biological significance, which is shared across multiple domains and covers
most pathological information. In this work, we propose a connectional style
contextual representation learning model (CS-CRL) to capture the intrinsic
pattern of the brain, used for multiple brain disease diagnosis. Specifically,
it has a vision transformer (ViT) encoder and leverages mask reconstruction as
the proxy task and Gram matrices to guide the representation of connectional
information. It facilitates the capture of global context and the aggregation
of features with biological plausibility. The results indicate that CS-CRL
achieves superior accuracy in multiple brain disease diagnosis tasks across six
datasets and three diseases and outperforms state-of-the-art models.
Furthermore, we demonstrate that CS-CRL captures more brain-network-like
properties, better aggregates features, is easier to optimize and is more
robust to noise, which explains its superiority in theory. Our source code will
be released soon.

提出一种基于连接方式的上下文表示学习模型 (CS-CRL)，用于多种脑病的诊断，特别是具有视觉变换 (ViT) 编码器和掩码重建作为代理任务和格拉姆矩阵来指导连通信息的表示，能够更好地聚合特征和提取全局上下文。实验结果表明，其在六个数据集和三种脑病的诊断任务中取得了卓越的准确性，并且胜过了现有的最先进模型。

基于连接式引导的上下文表示学习用于脑疾病诊断

Connectional-Style-Guided Contextual Representation Learning for Brain  Disease Diagnosis

Brain graph representation learning serves as the fundamental technique for
brain diseases diagnosis. Great efforts from both the academic and industrial
communities have been devoted to brain graph representation learning in recent
years. The isomorphic neural network (IsoNN) introduced recently can
automatically learn the existence of sub-graph patterns in brain graphs, which
is also the state-of-the-art brain graph representation learning method by this
context so far. However, IsoNN fails to capture the orientations of sub-graph
patterns, which may render the learned representations to be useless for many
cases. In this paper, we propose a new Iso-CapsNet (Isomorphic Capsule Net)
model by introducing the graph isomorphic capsules for effective brain graph
representation learning. Based on the capsule dynamic routing, besides the
subgraph pattern existence confidence scores, Iso-CapsNet can also learn other
sub-graph rich properties, including position, size and orientation, for
calculating the class-wise digit capsules. We have compared Iso-CapsNet with
both classic and state-of-the-art brain graph representation approaches with
extensive experiments on four brain graph benchmark datasets. The experimental
results also demonstrate the effectiveness of Iso-CapsNet, which can
out-perform the baseline methods with significant improvements.

该研究提出了一种名为 Iso-CapsNet 的新的神经网络模型，通过引入图同构胶囊，实现了对于脑图表征的有效学习。这个新的方法可以同时学习子图的存在、位置、大小和方向等特性，提高了基于脑图的诊断准确性。