Federated Learning (FL) aims at unburdening the training of deep models by
distributing computation across multiple devices (clients) while safeguarding
data privacy. On top of that, Federated Continual Learning (FCL) also accounts
for data distribution evolving over time, mirroring the dynamic nature of
real-world environments. In this work, we shed light on the Incremental and
Federated biases that naturally emerge in FCL. While the former is a known
problem in Continual Learning, stemming from the prioritization of recently
introduced classes, the latter (i.e., the bias towards local distributions)
remains relatively unexplored. Our proposal constrains both biases in the last
layer by efficiently fine-tuning a pre-trained backbone using learnable
prompts, resulting in clients that produce less biased representations and more
biased classifiers. Therefore, instead of solely relying on parameter
aggregation, we also leverage generative prototypes to effectively balance the
predictions of the global model. Our method improves on the current State Of
The Art, providing an average increase of +7.9% in accuracy.

该研究提出了一种通过使用可学习的提示来约束增量和联邦偏差的方法，从而提高全局模型的预测准确性，并在平均准确率上提供了 7.9% 的增加。

用层次生成原型来减少联邦式增量学习中的偏见

Reducing Bias in Federated Class-Incremental Learning with Hierarchical  Generative Prototypes

Contrastive Vision-Language Pre-training, known as CLIP, has shown promising
effectiveness in addressing downstream image recognition tasks. However, recent
works revealed that the CLIP model can be implanted with a downstream-oriented
backdoor. On downstream tasks, one victim model performs well on clean samples
but predicts a specific target class whenever a specific trigger is present.
For injecting a backdoor, existing attacks depend on a large amount of
additional data to maliciously fine-tune the entire pre-trained CLIP model,
which makes them inapplicable to data-limited scenarios. In this work,
motivated by the recent success of learnable prompts, we address this problem
by injecting a backdoor into the CLIP model in the prompt learning stage. Our
method named BadCLIP is built on a novel and effective mechanism in backdoor
attacks on CLIP, i.e., influencing both the image and text encoders with the
trigger. It consists of a learnable trigger applied to images and a
trigger-aware context generator, such that the trigger can change text features
via trigger-aware prompts, resulting in a powerful and generalizable attack.
Extensive experiments conducted on 11 datasets verify that the clean accuracy
of BadCLIP is similar to those of advanced prompt learning methods and the
attack success rate is higher than 99% in most cases. BadCLIP is also
generalizable to unseen classes, and shows a strong generalization capability
under cross-dataset and cross-domain settings.

本研究提出了一种名为 BadCLIP 的方法，针对 CLIP 模型在 prompt 学习阶段注入后门，通过影响图像和文本编码器的方式生成可触发的上下文，实现对图片和文本特征的修改，从而进行高效且具有泛化能力的攻击。实验证实，BadCLIP 在多个数据集上表现出与先进的 prompt 学习方法相近的准确性，攻击成功率在大多数情况下超过 99%；BadCLIP 在未知类别上也具有泛化能力，并在跨数据集和跨领域场景下展现了强大的泛化能力。

BadCLIP：针对 CLIP 的触发器感知提示学习的后门攻击

BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP

Multimodal Sentiment Analysis (MSA) has been a popular topic in natural
language processing nowadays, at both sentence and aspect level. However, the
existing approaches almost require large-size labeled datasets, which bring
about large consumption of time and resources. Therefore, it is practical to
explore the method for few-shot sentiment analysis in cross-modalities.
Previous works generally execute on textual modality, using the prompt-based
methods, mainly two types: hand-crafted prompts and learnable prompts. The
existing approach in few-shot multi-modality sentiment analysis task has
utilized both methods, separately. We further design a hybrid pattern that can
combine one or more fixed hand-crafted prompts and learnable prompts and
utilize the attention mechanisms to optimize the prompt encoder. The
experiments on both sentence-level and aspect-level datasets prove that we get
a significant outperformance.

本文提出了一种混合模式的近似零样本多模态情感分析方法，结合手工提示和可学习提示，利用注意力机制优化提示编码器，在句子水平和方面水平数据集上取得了显著的性能优势。