Recently, deep cross-modal hashing has gained increasing attention. However,
in many practical cases, data are distributed and cannot be collected due to
privacy concerns, which greatly reduces the cross-modal hashing performance on
each client. And due to the problems of statistical heterogeneity, model
heterogeneity, and forcing each client to accept the same parameters, applying
federated learning to cross-modal hash learning becomes very tricky. In this
paper, we propose a novel method called prototype-based layered federated
cross-modal hashing. Specifically, the prototype is introduced to learn the
similarity between instances and classes on server, reducing the impact of
statistical heterogeneity (non-IID) on different clients. And we monitor the
distance between local and global prototypes to further improve the
performance. To realize personalized federated learning, a hypernetwork is
deployed on server to dynamically update different layers' weights of local
model. Experimental results on benchmark datasets show that our method
outperforms state-of-the-art methods.

该论文提出了一种名为基于原型的分层联邦跨模式哈希的新方法来解决在不同客户端上运行时由于数据隐私及其他原因导致的模型异构性问题，提高模型性能。

一种基于原型的分层联邦跨模态哈希

Prototype-Based Layered Federated Cross-Modal Hashing

Due to its low storage cost and fast query speed, cross-modal hashing (CMH)
has been widely used for similarity search in multimedia retrieval
applications. However, almost all existing CMH methods are based on
hand-crafted features which might not be optimally compatible with the
hash-code learning procedure. As a result, existing CMH methods with
handcrafted features may not achieve satisfactory performance. In this paper,
we propose a novel cross-modal hashing method, called deep crossmodal hashing
(DCMH), by integrating feature learning and hash-code learning into the same
framework. DCMH is an end-to-end learning framework with deep neural networks,
one for each modality, to perform feature learning from scratch. Experiments on
two real datasets with text-image modalities show that DCMH can outperform
other baselines to achieve the state-of-the-art performance in cross-modal
retrieval applications.

本文提出了一种名为深度跨模态哈希（DCMH）的方法，通过将特征学习和哈希码学习整合到同一框架中来解决现有跨模态哈希方法中手工制作的特征与哈希码学习程序不兼容的问题，并使用基于神经网络的端到端学习方法，取得了在跨模态检索中的最优性能。