The advent of generative AI models has revolutionized digital content
creation, yet it introduces challenges in maintaining copyright integrity due
to generative parroting, where models mimic their training data too closely.
Our research presents a novel approach to tackle this issue by employing an
overfitted Masked Autoencoder (MAE) to detect such parroted samples
effectively. We establish a detection threshold based on the mean loss across
the training dataset, allowing for the precise identification of parroted
content in modified datasets. Preliminary evaluations demonstrate promising
results, suggesting our method's potential to ensure ethical use and enhance
the legal compliance of generative models.

通过采用过拟合的蒙面自动编码器（MAE）来有效检测生成式模型中的模仿样本，我们的研究提出了一种应对版权完整性问题的新方法。基于训练数据集上的平均损失建立检测阈值，从而可以精确识别修改后数据集中的模仿内容。初步评估表明我们的方法具有潜力，可以确保伦理使用并增强生成式模型的合法合规性。

通过过度拟合掩码自编码器检测生成抄袭

Detecting Generative Parroting through Overfitting Masked Autoencoders

Masked Autoencoder~(MAE) is a prevailing self-supervised learning method that
achieves promising results in model pre-training. However, when the various
downstream tasks have data distributions different from the pre-training data,
the semantically irrelevant pre-training information might result in negative
transfer, impeding MAE's scalability. To address this issue, we propose a novel
MAE-based pre-training paradigm, Mixture of Cluster-conditional Experts (MoCE),
which can be trained once but provides customized pre-training models for
diverse downstream tasks. Different from the mixture of experts (MoE), our MoCE
trains each expert only with semantically relevant images by using
cluster-conditional gates. Thus, each downstream task can be allocated to its
customized model pre-trained with data most similar to the downstream data.
Experiments on a collection of 11 downstream tasks show that MoCE outperforms
the vanilla MAE by 2.45\% on average. It also obtains new state-of-the-art
self-supervised learning results on detection and segmentation.

提出了一种新的基于混合的聚类条件专家（MoCE）的 MAE 预训练范式，为不同的下游任务提供定制化的预训练模型，通过使用聚类条件门将每个专家仅与语义相关的图像进行训练，从而克服了 MAE 可扩展性中的负迁移问题。

基于混合簇条件专家的任务定制化遮蔽自编码器

Task-customized Masked AutoEncoder via Mixture of Cluster-conditional  Experts

Medical time series data are indispensable in healthcare, providing critical
insights for disease diagnosis, treatment planning, and patient management. The
exponential growth in data complexity, driven by advanced sensor technologies,
has presented challenges related to data labeling. Self-supervised learning
(SSL) has emerged as a transformative approach to address these challenges,
eliminating the need for extensive human annotation. In this study, we
introduce a novel framework for Medical Time Series Representation Learning,
known as MTS-LOF. MTS-LOF leverages the strengths of contrastive learning and
Masked Autoencoder (MAE) methods, offering a unique approach to representation
learning for medical time series data. By combining these techniques, MTS-LOF
enhances the potential of healthcare applications by providing more
sophisticated, context-rich representations. Additionally, MTS-LOF employs a
multi-masking strategy to facilitate occlusion-invariant feature learning. This
approach allows the model to create multiple views of the data by masking
portions of it. By minimizing the discrepancy between the representations of
these masked patches and the fully visible patches, MTS-LOF learns to capture
rich contextual information within medical time series datasets. The results of
experiments conducted on diverse medical time series datasets demonstrate the
superiority of MTS-LOF over other methods. These findings hold promise for
significantly enhancing healthcare applications by improving representation
learning. Furthermore, our work delves into the integration of joint-embedding
SSL and MAE techniques, shedding light on the intricate interplay between
temporal and structural dependencies in healthcare data. This understanding is
crucial, as it allows us to grasp the complexities of healthcare data analysis.

医学时间序列数据的表征学习的新框架 MTS-LOF 借鉴对比学习和掩蔽自编码器方法的优势，通过多掩蔽策略学习医学时间序列数据中的丰富上下文信息，并证明优于其他方法，可显著增强医疗应用的表征学习。同样，该研究对联合嵌入自监督学习和自编码器技术的集成进行了深入探讨，揭示了医疗数据中时间和结构依赖性之间的错综复杂关系。