With the development of multimedia applications, multimodal recommendations
are playing an essential role, as they can leverage rich contexts beyond user
interactions. Existing methods mainly regard multimodal information as an
auxiliary, using them to help learn ID features; however, there exist semantic
gaps among multimodal content features and ID features, for which directly
using multimodal information as an auxiliary would lead to misalignment in
representations of users and items. In this paper, we first systematically
investigate the misalignment issue in multimodal recommendations, and propose a
solution named AlignRec. In AlignRec, the recommendation objective is
decomposed into three alignments, namely alignment within contents, alignment
between content and categorical ID, and alignment between users and items. Each
alignment is characterized by a specific objective function and is integrated
into our multimodal recommendation framework. To effectively train our
AlignRec, we propose starting from pre-training the first alignment to obtain
unified multimodal features and subsequently training the following two
alignments together with these features as input. As it is essential to analyze
whether each multimodal feature helps in training, we design three new classes
of metrics to evaluate intermediate performance. Our extensive experiments on
three real-world datasets consistently verify the superiority of AlignRec
compared to nine baselines. We also find that the multimodal features generated
by AlignRec are better than currently used ones, which are to be open-sourced.

通过系统调查多模态推荐中的对齐问题，本文提出了一种名为 AlignRec 的解决方案，由内容内部对齐、内容与分类 ID 之间的对齐，以及用户与项目之间的对齐这三个目标函数组成，将其融入到我们的多模态推荐框架中，并通过实验证明了 AlignRec 相对于其他九种基线方法的优越性。

多模态推荐的对齐和训练框架

An Aligning and Training Framework for Multimodal Recommendations

Most state-of-the-art methods for medical image segmentation adopt the
encoder-decoder architecture. However, this U-shaped framework still has
limitations in capturing the non-local multi-scale information with a simple
skip connection. To solve the problem, we firstly explore the potential
weakness of skip connections in U-Net on multiple segmentation tasks, and find
that i) not all skip connections are useful, each skip connection has different
contribution; ii) the optimal combinations of skip connections are different,
relying on the specific datasets. Based on our findings, we propose a new
segmentation framework, named UDTransNet, to solve three semantic gaps in
U-Net. Specifically, we propose a Dual Attention Transformer (DAT) module for
capturing the channel- and spatial-wise relationships to better fuse the
encoder features, and a Decoder-guided Recalibration Attention (DRA) module for
effectively connecting the DAT tokens and the decoder features to eliminate the
inconsistency. Hence, both modules establish a learnable connection to solve
the semantic gaps between the encoder and the decoder, which leads to a
high-performance segmentation model for medical images. Comprehensive
experimental results indicate that our UDTransNet produces higher evaluation
scores and finer segmentation results with relatively fewer parameters over the
state-of-the-art segmentation methods on different public datasets. Code:
this https URL

在医学图像分割中，通过探索 U-Net 中 skip 连接的潜在弱点，我们提出了 UDTransNet 框架，使用 Dual Attention Transformer (DAT) 和 Decoder-guided Recalibration Attention (DRA) 模块来解决编码器和解码器之间的语义差距，从而提高医学图像的分割效果。

通过可学习的跳跃连接缩小 U-Net 中的语义差距：以医学图像分割为例

Narrowing the semantic gaps in U-Net with learnable skip connections:  The case of medical image segmentation

Autonomous Driving (AD) systems rely on AI components to make safety and
correct driving decisions. Unfortunately, today's AI algorithms are known to be
generally vulnerable to adversarial attacks. However, for such AI
component-level vulnerabilities to be semantically impactful at the system
level, it needs to address non-trivial semantic gaps both (1) from the
system-level attack input spaces to those at AI component level, and (2) from
AI component-level attack impacts to those at the system level. In this paper,
we define such research space as semantic AI security as opposed to generic AI
security. Over the past 5 years, increasingly more research works are performed
to tackle such semantic AI security challenges in AD context, which has started
to show an exponential growth trend.
In this paper, we perform the first systematization of knowledge of such
growing semantic AD AI security research space. In total, we collect and
analyze 53 such papers, and systematically taxonomize them based on research
aspects critical for the security field. We summarize 6 most substantial
scientific gaps observed based on quantitative comparisons both vertically
among existing AD AI security works and horizontally with security works from
closely-related domains. With these, we are able to provide insights and
potential future directions not only at the design level, but also at the
research goal, methodology, and community levels. To address the most critical
scientific methodology-level gap, we take the initiative to develop an
open-source, uniform, and extensible system-driven evaluation platform, named
PASS, for the semantic AD AI security research community. We also use our
implemented platform prototype to showcase the capabilities and benefits of
such a platform using representative semantic AD AI attacks.

对自动驾驶（AD）系统中语义 AI 安全的研究领域进行了系统化概括，总共收集和分析了 53 篇论文，概括出 6 个最实质性的科学空白。为了解决最关键的科学方法层面的空缺，我们发起了开源、统一、可扩展的系统驱动评估平台 PASS 的开发，并使用我们实现的平台原型展示了这种平台的能力和优势。