We introduce SHARCS for adaptive inference that takes into account the
hardness of input samples. SHARCS can train a router on any transformer
network, enabling the model to direct different samples to sub-networks with
varying widths. Our experiments demonstrate that: (1) SHARCS outperforms or
complements existing per-sample adaptive inference methods across various
classification tasks in terms of accuracy vs. FLOPs; (2) SHARCS generalizes
across different architectures and can be even applied to compressed and
efficient transformer encoders to further improve their efficiency; (3) SHARCS
can provide a 2 times inference speed up at an insignificant drop in accuracy.

SHARCS 是一种自适应推理方法，通过考虑输入样本的难度，训练了一个路由器来将不同样本定向到具有不同宽度的子网络，实验证明，SHARCS 在准确性与 FLOPs 方面优于或补充了现有的逐样本自适应推理方法，能够泛化到不同的架构，甚至应用于压缩和高效的 Transformer 编码器以进一步提高其效率，并且能够在几乎不损失准确性的情况下提供 2 倍的推理加速。

SHARCS: 通过动态宽度子网络进行有效的 Transformer 路由

SHARCS: Efficient Transformers through Routing with Dynamic Width  Sub-networks

Multimodal learning is an essential paradigm for addressing complex
real-world problems, where individual data modalities are typically
insufficient to accurately solve a given modelling task. While various deep
learning approaches have successfully addressed these challenges, their
reasoning process is often opaque; limiting the capabilities for a principled
explainable cross-modal analysis and any domain-expert intervention. In this
paper, we introduce SHARCS (SHARed Concept Space) -- a novel concept-based
approach for explainable multimodal learning. SHARCS learns and maps
interpretable concepts from different heterogeneous modalities into a single
unified concept-manifold, which leads to an intuitive projection of
semantically similar cross-modal concepts. We demonstrate that such an approach
can lead to inherently explainable task predictions while also improving
downstream predictive performance. Moreover, we show that SHARCS can operate
and significantly outperform other approaches in practically significant
scenarios, such as retrieval of missing modalities and cross-modal
explanations. Our approach is model-agnostic and easily applicable to different
types (and number) of modalities, thus advancing the development of effective,
interpretable, and trustworthy multimodal approaches.

本文介绍了一种基于共享概念空间（SHARCS ）的可解释的多模态学习方法，该方法能够将来自不同异构模态的可解释的概念学习和映射到单个统一的概念流形中，从而得出具有内在解释能力的任务预测，同时提高下游预测性能，并在检索缺失模态和跨模态解释等方面快速且优异地完成任务。