Diffusion models (DMs) produce very detailed and high-quality images. Their
power results from extensive training on large amounts of data, usually scraped
from the internet without proper attribution or consent from content creators.
Unfortunately, this practice raises privacy and intellectual property concerns,
as DMs can memorize and later reproduce their potentially sensitive or
copyrighted training images at inference time. Prior efforts prevent this issue
by either changing the input to the diffusion process, thereby preventing the
DM from generating memorized samples during inference, or removing the
memorized data from training altogether. While those are viable solutions when
the DM is developed and deployed in a secure and constantly monitored
environment, they hold the risk of adversaries circumventing the safeguards and
are not effective when the DM itself is publicly released. To solve the
problem, we introduce NeMo, the first method to localize memorization of
individual data samples down to the level of neurons in DMs' cross-attention
layers. Through our experiments, we make the intriguing finding that in many
cases, single neurons are responsible for memorizing particular training
samples. By deactivating these memorization neurons, we can avoid the
replication of training data at inference time, increase the diversity in the
generated outputs, and mitigate the leakage of private and copyrighted data. In
this way, our NeMo contributes to a more responsible deployment of DMs.

通过定位跨注意力层中的神经元，我们引入了 NeMo 方法来解决扩散模型中的个别数据样本的记忆问题，从而避免了在推理过程中复制训练数据，增加了生成输出的多样性，并减少了私密和受版权保护数据的泄露，进而实现了更负责任的扩散模型的部署。

找到 NeMo: 在扩散模型中定位负责记忆的神经元

Finding NeMo: Localizing Neurons Responsible For Memorization in  Diffusion Models

Large language models (LLMs) and generative AI have played a transformative
role in computer research and applications. Controversy has arisen as to
whether these models output copyrighted data, which can occur if the data the
models are trained on is copyrighted. LLMs are built on the transformer neural
network architecture, which in turn relies on a mathematical computation called
Attention that uses the softmax function.
In this paper, we show that large language model training and optimization
can be seen as a softmax regression problem. We then establish a method of
efficiently performing softmax regression, in a way that prevents the
regression function from generating copyright data. This establishes a
theoretical method of training large language models in a way that avoids
generating copyright data.

利用训练大语言模型的理论方法，可以避免生成版权数据。

如何在大型语言模型的优化中保护版权数据？

How to Protect Copyright Data in Optimization of Large Language Models?

The legality of training language models (LMs) on copyrighted or otherwise
restricted data is under intense debate. However, as we show, model performance
significantly degrades if trained only on low-risk text (e.g., out-of-copyright
books or government documents), due to its limited size and domain coverage. We
present SILO, a new language model that manages this risk-performance tradeoff
during inference. SILO is built by (1) training a parametric LM on Open License
Corpus (OLC), a new corpus we curate with 228B tokens of public domain and
permissively licensed text and (2) augmenting it with a more general and easily
modifiable nonparametric datastore (e.g., containing copyrighted books or news)
that is only queried during inference. The datastore allows use of high-risk
data without training on it, supports sentence-level data attribution, and
enables data producers to opt out from the model by removing content from the
store. These capabilities can foster compliance with data-use regulations such
as the fair use doctrine in the United States and the GDPR in the European
Union. Our experiments show that the parametric LM struggles on domains not
covered by OLC. However, access to the datastore greatly improves out of domain
performance, closing 90% of the performance gap with an LM trained on the Pile,
a more diverse corpus with mostly high-risk text. We also analyze which
nonparametric approach works best, where the remaining errors lie, and how
performance scales with datastore size. Our results suggest that it is possible
to build high quality language models while mitigating their legal risk.

通过在开放许可的文本语料库（OLC）上训练参数化的语言模型，并在推断过程中使用包含有版权内容的数据存储库，SILO 模型能够在满足数据使用法规（如美国的公平使用原则和欧盟的 GDPR）的同时提高模型的性能。