With the rise of Large Language Models (LLMs) in recent years, new
opportunities are emerging, but also new challenges, and contamination is
quickly becoming critical. Business applications and fundraising in AI have
reached a scale at which a few percentage points gained on popular
question-answering benchmarks could translate into dozens of millions of
dollars, placing high pressure on model integrity. At the same time, it is
becoming harder and harder to keep track of the data that LLMs have seen; if
not impossible with closed-source models like GPT-4 and Claude-3 not divulging
any information on the training set. As a result, contamination becomes a
critical issue: LLMs' performance may not be reliable anymore, as the high
performance may be at least partly due to their previous exposure to the data.
This limitation jeopardizes the entire progress in the field of NLP, yet, there
remains a lack of methods on how to efficiently address contamination, or a
clear consensus on prevention, mitigation and classification of contamination.
In this paper, we survey all recent work on contamination with LLMs, and help
the community track contamination levels of LLMs by releasing an open-source
Python library named LLMSanitize implementing major contamination detection
algorithms, which link is: this https URL

LLMs 应用的规模不断扩大，但同时也带来了污染问题，而模型的完整性对业务应用和筹款至关重要。本文调查了 LLMs 污染问题的最新研究，并通过开源 Python 库 LLMSanitize 实现了主要污染检测算法，以帮助社区跟踪 LLMs 的污染水平。

LLM 污染程度调查及 LLMSanitize 库

How Much are LLMs Contaminated? A Comprehensive Survey and the  LLMSanitize Library

Diffusion models have gained prominence in the image domain for their
capabilities in data generation and transformation, achieving state-of-the-art
performance in various tasks in both image and audio domains. In the rapidly
evolving field of audio-based machine learning, safeguarding model integrity
and establishing data copyright are of paramount importance. This paper
presents the first watermarking technique applied to audio diffusion models
trained on mel-spectrograms. This offers a novel approach to the aforementioned
challenges. Our model excels not only in benign audio generation, but also
incorporates an invisible watermarking trigger mechanism for model
verification. This watermark trigger serves as a protective layer, enabling the
identification of model ownership and ensuring its integrity. Through extensive
experiments, we demonstrate that invisible watermark triggers can effectively
protect against unauthorized modifications while maintaining high utility in
benign audio generation tasks.

本文提出了首个应用于基于 mel 频谱图的音频扩散模型的水印技术，以解决音频机器学习领域中模型完整性和数据版权的问题。通过无形水印触发机制，该模型在良性音频生成中具有卓越性能，并能保护模型免受未经授权的修改。

音频生成扩散模型的隐形数字水印

Invisible Watermarking for Audio Generation Diffusion Models

Over-the-air federated learning (AirFL) allows devices to train a learning
model in parallel and synchronize their local models using over-the-air
computation. The integrity of AirFL is vulnerable due to the obscurity of the
local models aggregated over-the-air. This paper presents a novel framework to
balance the accuracy and integrity of AirFL, where multi-antenna devices and
base station (BS) are jointly optimized with a reconfigurable intelligent
surface (RIS). The key contributions include a new and non-trivial problem
jointly considering the model accuracy and integrity of AirFL, and a new
framework that transforms the problem into tractable subproblems. Under perfect
channel state information (CSI), the new framework minimizes the aggregated
model's distortion and retains the local models' recoverability by optimizing
the transmit beamformers of the devices, the receive beamformers of the BS, and
the RIS configuration in an alternating manner. Under imperfect CSI, the new
framework delivers a robust design of the beamformers and RIS configuration to
combat non-negligible channel estimation errors. As corroborated
experimentally, the novel framework can achieve comparable accuracy to the
ideal FL while preserving local model recoverability under perfect CSI, and
improve the accuracy when the number of receive antennas is small or moderate
under imperfect CSI.

本研究提出了一种新颖的框架，使用可重构智能表面（RIS）进行联合优化，解决了无线联邦学习（AirFl）中本地模型可靠性保持以及全局精度之间的权衡问题。该框架利用波束成形器和 CSI 信息通过联合优化解决了该问题。在实验中，该框架在仍能保持本地模型可靠性的情况下实现了与理想 FL 相当的准确性。