Diffusion models have achieved impressive results in generating diverse and
realistic data by employing multi-step denoising processes. However, the need
for accommodating significant variations in input noise at each time-step has
led to diffusion models requiring a large number of parameters for their
denoisers. We have observed that diffusion models effectively act as filters
for different frequency ranges at each time-step noise. While some previous
works have introduced multi-expert strategies, assigning denoisers to different
noise intervals, they overlook the importance of specialized operations for
high and low frequencies. For instance, self-attention operations are effective
at handling low-frequency components (low-pass filters), while convolutions
excel at capturing high-frequency features (high-pass filters). In other words,
existing diffusion models employ denoisers with the same architecture, without
considering the optimal operations for each time-step noise. To address this
limitation, we propose a novel approach called Multi-architecturE Multi-Expert
(MEME), which consists of multiple experts with specialized architectures
tailored to the operations required at each time-step interval. Through
extensive experiments, we demonstrate that MEME outperforms large competitors
in terms of both generation performance and computational efficiency.

本文提出了一种新颖的多专家方法 MEME，用于扩展扩散模型的过滤器以适应输入噪声的不同频率范围，并通过大量实验表明，MEME 在生成性能和计算效率方面优于传统方法

多架构多专家扩散模型

Multi-Architecture Multi-Expert Diffusion Models

The prevalence of memes on social media has created the need to sentiment
analyze their underlying meanings for censoring harmful content. Meme censoring
systems by machine learning raise the need for a semi-supervised learning
solution to take advantage of the large number of unlabeled memes available on
the internet and make the annotation process less challenging. Moreover, the
approach needs to utilize multimodal data as memes' meanings usually come from
both images and texts. This research proposes a multimodal semi-supervised
learning approach that outperforms other multimodal semi-supervised learning
and supervised learning state-of-the-art models on two datasets, the Multimedia
Automatic Misogyny Identification and Hateful Memes dataset. Building on the
insights gained from Contrastive Language-Image Pre-training, which is an
effective multimodal learning technique, this research introduces SemiMemes, a
novel training method that combines auto-encoder and classification task to
make use of the resourceful unlabeled data.

本篇研究提出一种多模态半监督学习方法，可用于探测社交媒体上的恶意图片，该方法结合了自动编码器和分类任务，通过 Contrastive Language-Image Pre-training 实现。在两个数据集上，该方法表现优于其他多模态半监督学习和有监督学习的现有模型。

SemiMemes: 一种用于多模态 Memes 分析的半监督学习方法

SemiMemes: A Semi-supervised Learning Approach for Multimodal Memes  Analysis

We consider the problem of communicating exogenous information by means of
Markov decision process trajectories. This setting, which we call a Markov
coding game (MCG), generalizes both source coding and a large class of
referential games. MCGs also isolate a problem that is important in
decentralized control settings in which cheap-talk is not available -- namely,
they require balancing communication with the associated cost of communicating.
We contribute a theoretically grounded approach to MCGs based on maximum
entropy reinforcement learning and minimum entropy coupling that we call MEME.
Due to recent breakthroughs in approximation algorithms for minimum entropy
coupling, MEME is not merely a theoretical algorithm, but can be applied to
practical settings. Empirically, we show both that MEME is able to outperform a
strong baseline on small MCGs and that MEME is able to achieve strong
performance on extremely large MCGs. To the latter point, we demonstrate that
MEME is able to losslessly communicate binary images via trajectories of
Cartpole and Pong, while simultaneously achieving the maximal or near maximal
expected returns, and that it is even capable of performing well in the
presence of actuator noise.

本文介绍了一种被称为 Markov 编码游戏 (MCG) 的方法来处理在分散控制环境下进行通信的问题，并且介绍了一种新的理论算法 MEME 来进行最大熵强化学习和最小熵耦合的平衡。同时进行的实验也表明了这种算法在解决小型和大型 MCG 问题时具有良好性能。