Current text-to-image diffusion models have achieved groundbreaking results
in image generation tasks. However, the unavoidable inclusion of sensitive
information during pre-training introduces significant risks such as copyright
infringement and privacy violations in the generated images. Machine Unlearning
(MU) provides a effective way to the sensitive concepts captured by the model,
has been shown to be a promising approach to addressing these issues.
Nonetheless, existing MU methods for concept erasure encounter two primary
bottlenecks: 1) generalization issues, where concept erasure is effective only
for the data within the unlearn set, and prompts outside the unlearn set often
still result in the generation of sensitive concepts; and 2) utility drop,
where erasing target concepts significantly degrades the model's performance.
To this end, this paper first proposes a concept domain correction framework
for unlearning concepts in diffusion models. By aligning the output domains of
sensitive concepts and anchor concepts through adversarial training, we enhance
the generalizability of the unlearning results. Secondly, we devise a
concept-preserving scheme based on gradient surgery. This approach alleviates
the parts of the unlearning gradient that contradict the relearning gradient,
ensuring that the process of unlearning minimally disrupts the model's
performance. Finally, extensive experiments validate the effectiveness of our
model, demonstrating our method's capability to address the challenges of
concept unlearning in diffusion models while preserving model utility.

通过对抗训练对齐敏感概念和锚定概念的输出领域，提出了一种用于扩展模型对概念擦除的概念领域校正框架，并且基于梯度修剪设计了一个能够减轻卸载梯度中与重新学习梯度相矛盾的部分的概念保留方案，从而在保留模型效用的同时解决了扩散模型中概念卸载的挑战问题。

通过概念领域修正和概念保持梯度，从扩散模型中去除概念

Unlearning Concepts in Diffusion Model via Concept Domain Correction and  Concept Preserving Gradient

Feedforward generalizable models for implicit shape reconstruction from
unoriented point cloud present multiple advantages, including high performance
and inference speed. However, they still suffer from generalization issues,
ranging from underfitting the input point cloud, to misrepresenting samples
outside of the training data distribution, or with toplogies unseen at
training. We propose here an efficient mechanism to remedy some of these
limitations at test time. We combine the inter-shape data prior of the network
with an intra-shape regularization prior of a Nystr\"om Kernel Ridge
Regression, that we further adapt by fitting its hyperprameters to the current
shape. The resulting shape function defined in a shape specific Reproducing
Kernel Hilbert Space benefits from desirable stability and efficiency
properties and grants a shape adaptive expressiveness-robustness trade-off. We
demonstrate the improvement obtained through our method with respect to
baselines and the state-of-the-art using synthetic and real data.

通过将网络的形状间数据先验与 Nyström 内部形状正则化先验相结合，在测试时提出了一种有效的机制来解决一些限制。通过使用综合内插核来适应当前形状的超参数，得到的形状函数在形状特定的再生核希尔伯特空间中具有稳定性和高效性，并提供形状自适应的表达力和鲁棒性的权衡。通过使用合成和真实数据，我们证明了我们的方法相对于基线和现有技术的改进。

提高通用性的隐式形状网络的鲁棒性：可调的非参数模型

Robustifying Generalizable Implicit Shape Networks with a Tunable  Non-Parametric Model

A significant effort has been made to train neural networks that replicate
algorithmic reasoning, but they often fail to learn the abstract concepts
underlying these algorithms. This is evidenced by their inability to generalize
to data distributions that are outside of their restricted training sets,
namely larger inputs and unseen data. We study these generalization issues at
the level of numerical subroutines that comprise common algorithms like
sorting, shortest paths, and minimum spanning trees. First, we observe that
transformer-based sequence-to-sequence models can learn subroutines like
sorting a list of numbers, but their performance rapidly degrades as the length
of lists grows beyond those found in the training set. We demonstrate that this
is due to attention weights that lose fidelity with longer sequences,
particularly when the input numbers are numerically similar. To address the
issue, we propose a learned conditional masking mechanism, which enables the
model to strongly generalize far outside of its training range with
near-perfect accuracy on a variety of algorithms. Second, to generalize to
unseen data, we show that encoding numbers with a binary representation leads
to embeddings with rich structure once trained on downstream tasks like
addition or multiplication. This allows the embedding to handle missing data by
faithfully interpolating numbers not seen during training.

通过研究数值子程序，该论文发现基于 transformer 的序列到序列模型可以学习排序等子程序，但当列表长度超出训练集时，性能迅速下降。作者提出了一种条件掩蔽机制来解决这个问题，并展示了使用二进制表示编码数字可以处理训练过程中未见数据的能力，从而取得更好的推广效果。