Multi-Channel Imaging (MCI) contains an array of challenges for encoding
useful feature representations not present in traditional images. For example,
images from two different satellites may both contain RGB channels, but the
remaining channels can be different for each imaging source. Thus, MCI models
must support a variety of channel configurations at test time. Recent work has
extended traditional visual encoders for MCI, such as Vision Transformers
(ViT), by supplementing pixel information with an encoding representing the
channel configuration. However, these methods treat each channel equally, i.e.,
they do not consider the unique properties of each channel type, which can
result in needless and potentially harmful redundancies in the learned
features. For example, if RGB channels are always present, the other channels
can focus on extracting information that cannot be captured by the RGB
channels. To this end, we propose DiChaViT, which aims to enhance the diversity
in the learned features of MCI-ViT models. This is achieved through a novel
channel sampling strategy that encourages the selection of more distinct
channel sets for training. Additionally, we employ regularization and
initialization techniques to increase the likelihood that new information is
learned from each channel. Many of our improvements are architecture agnostic
and could be incorporated into new architectures as they are developed.
Experiments on both satellite and cell microscopy datasets, CHAMMI, JUMP-CP,
and So2Sat, report DiChaViT yields a 1.5-5.0% gain over the state-of-the-art.

通过使用一种新的通道采样策略来增强 MCI-ViT 模型的特征多样性，DiChaViT 方法在卫星和细胞显微图像数据集上实验证明其超过现有技术 1.5-5.0% 的性能提升。

增强特征多样性提升通道自适应视觉变换器

Enhancing Feature Diversity Boosts Channel-Adaptive Vision Transformers

Integrating the brain structural and functional connectivity features is of
great significance in both exploring brain science and analyzing cognitive
impairment clinically. However, it remains a challenge to effectively fuse
structural and functional features in exploring the brain network. In this
paper, a novel brain structure-function fusing-representation learning (BSFL)
model is proposed to effectively learn fused representation from diffusion
tensor imaging (DTI) and resting-state functional magnetic resonance imaging
(fMRI) for mild cognitive impairment (MCI) analysis. Specifically, the
decomposition-fusion framework is developed to first decompose the feature
space into the union of the uniform and the unique spaces for each modality,
and then adaptively fuse the decomposed features to learn MCI-related
representation. Moreover, a knowledge-aware transformer module is designed to
automatically capture local and global connectivity features throughout the
brain. Also, a uniform-unique contrastive loss is further devised to make the
decomposition more effective and enhance the complementarity of structural and
functional features. The extensive experiments demonstrate that the proposed
model achieves better performance than other competitive methods in predicting
and analyzing MCI. More importantly, the proposed model could be a potential
tool for reconstructing unified brain networks and predicting abnormal
connections during the degenerative processes in MCI.

本研究提出一种新的脑结构 - 功能融合表征学习模型，它可用于从 DTI 和 fMRI 中学习融合表示，以分析轻度认知障碍 (MCI)。该模型通过将特征空间分解为每种模态的统一和独特空间的并集，并自适应融合分解功能，来有效融合脑的结构和功能特征，同时设计了一种知识感知变压器模块来捕获全局和局部连接特征。实验表明，该模型在预测和分析 MCI 方面的表现优于其他竞争方法，有望成为重建统一脑网络并预测 MCI 退化的异常连接的潜在工具。

使用对抗分解变分自编码器的脑结构功能融合学习，分析 MCI

Brain Structure-Function Fusing Representation Learning using  Adversarial Decomposed-VAE for Analyzing MCI

Objective: We develop a deep learning framework based on the pre-trained
Bidirectional Encoder Representations from Transformers (BERT) model using
unstructured clinical notes from electronic health records (EHRs) to predict
the risk of disease progression from Mild Cognitive Impairment (MCI) to
Alzheimer's Disease (AD). Materials and Methods: We identified 3657 patients
diagnosed with MCI together with their progress notes from Northwestern
Medicine Enterprise Data Warehouse (NMEDW) between 2000-2020. The progress
notes no later than the first MCI diagnosis were used for the prediction. We
first preprocessed the notes by deidentification, cleaning and splitting, and
then pretrained a BERT model for AD (AD-BERT) based on the publicly available
Bio+Clinical BERT on the preprocessed notes. The embeddings of all the sections
of a patient's notes processed by AD-BERT were combined by MaxPooling to
compute the probability of MCI-to-AD progression. For replication, we conducted
a similar set of experiments on 2563 MCI patients identified at Weill Cornell
Medicine (WCM) during the same timeframe. Results: Compared with the 7 baseline
models, the AD-BERT model achieved the best performance on both datasets, with
Area Under receiver operating characteristic Curve (AUC) of 0.8170 and F1 score
of 0.4178 on NMEDW dataset and AUC of 0.8830 and F1 score of 0.6836 on WCM
dataset. Conclusion: We developed a deep learning framework using BERT models
which provide an effective solution for prediction of MCI-to-AD progression
using clinical note analysis.

该研究使用医疗电子记录中未结构化的临床笔记数据，采用基于 BERT 预训练模型的深度学习框架，预测从轻度认知障碍到阿尔茨海默症的疾病进展风险。