Multimodal learning is an essential paradigm for addressing complex
real-world problems, where individual data modalities are typically
insufficient to accurately solve a given modelling task. While various deep
learning approaches have successfully addressed these challenges, their
reasoning process is often opaque; limiting the capabilities for a principled
explainable cross-modal analysis and any domain-expert intervention. In this
paper, we introduce SHARCS (SHARed Concept Space) -- a novel concept-based
approach for explainable multimodal learning. SHARCS learns and maps
interpretable concepts from different heterogeneous modalities into a single
unified concept-manifold, which leads to an intuitive projection of
semantically similar cross-modal concepts. We demonstrate that such an approach
can lead to inherently explainable task predictions while also improving
downstream predictive performance. Moreover, we show that SHARCS can operate
and significantly outperform other approaches in practically significant
scenarios, such as retrieval of missing modalities and cross-modal
explanations. Our approach is model-agnostic and easily applicable to different
types (and number) of modalities, thus advancing the development of effective,
interpretable, and trustworthy multimodal approaches.

本文介绍了一种基于共享概念空间（SHARCS ）的可解释的多模态学习方法，该方法能够将来自不同异构模态的可解释的概念学习和映射到单个统一的概念流形中，从而得出具有内在解释能力的任务预测，同时提高下游预测性能，并在检索缺失模态和跨模态解释等方面快速且优异地完成任务。

SHARCS：可解释多模态学习的共享概念空间

SHARCS: Shared Concept Space for Explainable Multimodal Learning

A long-held objective in AI is to build systems that understand concepts in a
humanlike way. Setting aside the difficulty of building such a system, even
trying to evaluate one is a challenge, due to present-day AI's relative opacity
and its proclivity for finding shortcut solutions. This is exacerbated by
humans' tendency to anthropomorphize, assuming that a system that can recognize
one instance of a concept must also understand other instances, as a human
would. In this paper, we argue that understanding a concept requires the
ability to use it in varied contexts. Accordingly, we propose systematic
evaluations centered around concepts, by probing a system's ability to use a
given concept in many different instantiations. We present case studies of such
an evaluations on two domains -- RAVEN (inspired by Raven's Progressive
Matrices) and the Abstraction and Reasoning Corpus (ARC) -- that have been used
to develop and assess abstraction abilities in AI systems. Our concept-based
approach to evaluation reveals information about AI systems that conventional
test sets would have left hidden.

本文提出了一种基于概念的评估方法，通过探究一个系统在多个实例中应用给定概念的能力来评估。作者以 RAVEN 和 ARC 为例进行了案例研究。研究发现，这种基于概念的评估方法揭示了传统测试集所隐藏的 AI 系统的信息。

评估概念抽象基准的理解能力

Evaluating Understanding on Conceptual Abstraction Benchmarks

To solve complex real-world problems, heuristics and concept-based approaches
can be used in order to incorporate information into the problem. In this
study, a concept-based approach called variable functioning Fx is introduced to
reduce the optimization variables and narrow down the search space. In this
method, the relationships among one or more subset of variables are defined
with functions using information prior to optimization; thus, instead of
modifying the variables in the search process, the function variables are
optimized. By using problem structure analysis technique and engineering expert
knowledge, the $Fx$ method is used to enhance the steel frame design
optimization process as a complex real-world problem. The proposed approach is
coupled with particle swarm optimization and differential evolution algorithms
and used for three case studies. The algorithms are applied to optimize the
case studies by considering the relationships among column cross-section areas.
The results show that $Fx$ can significantly improve both the convergence rate
and the final design of a frame structure, even if it is only used for seeding.

本研究介绍了一种名为变量功能 Fx 的基于概念的方法，通过事先定义一些变量之间的函数关系来缩小搜索空间，用于优化复杂现实世界问题的钢框架设计，结果表明 Fx 方法可以显著提高结构设计的收敛速度和最终设计。