Multimodal Large Language Models (MLLMs) demonstrate remarkable capabilities
that increasingly influence various aspects of our daily lives, constantly
defining the new boundary of Artificial General Intelligence (AGI). Image
modalities, enriched with profound semantic information and a more continuous
mathematical nature compared to other modalities, greatly enhance the
functionalities of MLLMs when integrated. However, this integration serves as a
double-edged sword, providing attackers with expansive vulnerabilities to
exploit for highly covert and harmful attacks. The pursuit of reliable AI
systems like powerful MLLMs has emerged as a pivotal area of contemporary
research. In this paper, we endeavor to demostrate the multifaceted risks
associated with the incorporation of image modalities into MLLMs. Initially, we
delineate the foundational components and training processes of MLLMs.
Subsequently, we construct a threat model, outlining the security
vulnerabilities intrinsic to MLLMs. Moreover, we analyze and summarize existing
scholarly discourses on MLLMs' attack and defense mechanisms, culminating in
suggestions for the future research on MLLM security. Through this
comprehensive analysis, we aim to deepen the academic understanding of MLLM
security challenges and propel forward the development of trustworthy MLLM
systems.

多模态大型语言模型（MLLMs）的整合增强了其功能，但也带来了安全漏洞，本研究旨在分析并总结 MLLMs 的攻击和防御机制，并提出未来研究的建议，以深化对 MLLM 安全挑战的学术理解和发展可信赖的 MLLM 系统。

狂放的伊卡洛斯：多模态大语言模型安全中图像输入的潜在危险调研

Unbridled Icarus: A Survey of the Potential Perils of Image Inputs in  Multimodal Large Language Model Security

Anomaly detection (AD) aims at detecting abnormal samples that deviate from
the expected normal patterns. Generally, it can be trained on merely normal
data without the requirement for abnormal samples, and thereby plays an
important role in the recognition of rare diseases and health screening in the
medical domain. Despite numerous related studies, we observe a lack of a fair
and comprehensive evaluation, which causes some ambiguous conclusions and
hinders the development of this field. This paper focuses on building a
benchmark with unified implementation and comparison to address this problem.
In particular, seven medical datasets with five image modalities, including
chest X-rays, brain MRIs, retinal fundus images, dermatoscopic images, and
histopathology whole slide images are organized for extensive evaluation.
Twenty-seven typical AD methods, including reconstruction and self-supervised
learning-based methods, are involved in comparison of image-level anomaly
classification and pixel-level anomaly segmentation. Furthermore, we for the
first time formally explore the effect of key components in existing methods,
clearly revealing unresolved challenges and potential future directions. The
datasets and code are available at
https://github.com/caiyu6666/MedIAnomaly.

该研究论文旨在建立一个统一实施和比较的基准，对医学领域的七个包括胸部 X 射线、脑部磁共振、视网膜底像、皮肤镜像和组织病理全切片图像在内的医学数据集进行广泛评估，并比较了二十七种典型的异常检测方法，首次对现有方法中的关键组件进行了正式探索，明确了未解决的挑战和潜在的未来方向。

MedIAnomaly：医学图像异常检测的比较研究

MedIAnomaly: A comparative study of anomaly detection in medical images

We present a semi-supervised domain adaptation framework for brain vessel
segmentation from different image modalities. Existing state-of-the-art methods
focus on a single modality, despite the wide range of available cerebrovascular
imaging techniques. This can lead to significant distribution shifts that
negatively impact the generalization across modalities. By relying on annotated
angiographies and a limited number of annotated venographies, our framework
accomplishes image-to-image translation and semantic segmentation, leveraging a
disentangled and semantically rich latent space to represent heterogeneous data
and perform image-level adaptation from source to target domains. Moreover, we
reduce the typical complexity of cycle-based architectures and minimize the use
of adversarial training, which allows us to build an efficient and intuitive
model with stable training. We evaluate our method on magnetic resonance
angiographies and venographies. While achieving state-of-the-art performance in
the source domain, our method attains a Dice score coefficient in the target
domain that is only 8.9% lower, highlighting its promising potential for robust
cerebrovascular image segmentation across different modalities.

基于半监督域自适应的脑血管分割方法，通过图像到图像的转换和语义分割，利用一个分离和语义丰富的潜在空间来表示异构数据并在不同模态之间进行图像级别的自适应，实现了高效且稳定的模型训练，并在不同模态下取得了同源领域最佳性能与目标领域 8.9% 的 Dice 值差距，具有可靠的脑血管图像分割潜力。