Recently, deep cross-modal hashing has gained increasing attention. However,
in many practical cases, data are distributed and cannot be collected due to
privacy concerns, which greatly reduces the cross-modal hashing performance on
each client. And due to the problems of statistical heterogeneity, model
heterogeneity, and forcing each client to accept the same parameters, applying
federated learning to cross-modal hash learning becomes very tricky. In this
paper, we propose a novel method called prototype-based layered federated
cross-modal hashing. Specifically, the prototype is introduced to learn the
similarity between instances and classes on server, reducing the impact of
statistical heterogeneity (non-IID) on different clients. And we monitor the
distance between local and global prototypes to further improve the
performance. To realize personalized federated learning, a hypernetwork is
deployed on server to dynamically update different layers' weights of local
model. Experimental results on benchmark datasets show that our method
outperforms state-of-the-art methods.

该论文提出了一种名为基于原型的分层联邦跨模式哈希的新方法来解决在不同客户端上运行时由于数据隐私及其他原因导致的模型异构性问题，提高模型性能。