Prompt learning has emerged as an effective and data-efficient technique in
large Vision-Language Models (VLMs). However, when adapting VLMs to specialized
domains such as remote sensing and medical imaging, domain prompt learning
remains underexplored. While large-scale domain-specific foundation models can
help tackle this challenge, their concentration on a single vision level makes
it challenging to prompt both vision and language modalities. To overcome this,
we propose to leverage domain-specific knowledge from domain-specific
foundation models to transfer the robust recognition ability of VLMs from
generalized to specialized domains, using quaternion networks. Specifically,
the proposed method involves using domain-specific vision features from
domain-specific foundation models to guide the transformation of generalized
contextual embeddings from the language branch into a specialized space within
the quaternion networks. Moreover, we present a hierarchical approach that
generates vision prompt features by analyzing intermodal relationships between
hierarchical language prompt features and domain-specific vision features. In
this way, quaternion networks can effectively mine the intermodal relationships
in the specific domain, facilitating domain-specific vision-language
contrastive learning. Extensive experiments on domain-specific datasets show
that our proposed method achieves new state-of-the-art results in prompt
learning.

通过使用四元数网络，利用专用领域模型中的领域特定视觉特征引导通用上下文嵌入的转换来实现从通用化到专用化领域的有效识别能力转移，同时采用层次化方法生成视觉提示特征并分析层次化语言提示特征和领域专用视觉特征之间的互模态关系，从而实现在特定领域中有效挖掘互模态关系，促进领域特定的视觉语言对比学习，并通过在特定领域的数据集上进行广泛实验，证明我们的方法在提示学习方面取得了最新的最好的结果。

域提示学习及其四元数网络

Domain Prompt Learning with Quaternion Networks

The field of deep learning has seen significant advancement in recent years.
However, much of the existing work has been focused on real-valued numbers.
Recent work has shown that a deep learning system using the complex numbers can
be deeper for a fixed parameter budget compared to its real-valued counterpart.
In this work, we explore the benefits of generalizing one step further into the
hyper-complex numbers, quaternions specifically, and provide the architecture
components needed to build deep quaternion networks. We develop the theoretical
basis by reviewing quaternion convolutions, developing a novel quaternion
weight initialization scheme, and developing novel algorithms for quaternion
batch-normalization. These pieces are tested in a classification model by
end-to-end training on the CIFAR-10 and CIFAR-100 data sets and a segmentation
model by end-to-end training on the KITTI Road Segmentation data set. These
quaternion networks show improved convergence compared to real-valued and
complex-valued networks, especially on the segmentation task, while having
fewer parameters

本文介绍了如何使用四元数构建深度网络，通过在 CIFAR-10 和 CIFAR-100 数据集上进行端到端训练以及在 KITTI Road Segmentation 数据集上进行端到端训练，我们证明了四元数网络比实数网络和复数网络具有更好的收敛性能，在分割任务中表现尤为突出。