Large Language Models (LLMs) have highlighted the necessity of effective
unlearning mechanisms to comply with data regulations and ethical AI practices.
LLM unlearning aims at removing undesired data influences and associated model
capabilities without compromising utility out of the scope of unlearning. While
interest in studying LLM unlearning is growing,the impact of the optimizer
choice for LLM unlearning remains under-explored. In this work, we shed light
on the significance of optimizer selection in LLM unlearning for the first
time, establishing a clear connection between {second-order optimization} and
influence unlearning (a classical approach using influence functions to update
the model for data influence removal). This insight propels us to develop a
second-order unlearning framework, termed SOUL, built upon the second-order
clipped stochastic optimization (Sophia)-based LLM training method. SOUL
extends the static, one-shot model update using influence unlearning to a
dynamic, iterative unlearning process. Our extensive experiments show that SOUL
consistently outperforms conventional first-order methods across various
unlearning tasks, models, and metrics, suggesting the promise of second-order
optimization in providing a scalable and easily implementable solution for LLM
unlearning.

通过选择优化器，本研究揭示了第二阶优化在大型语言模型的遗忘中的重要性，并基于该观察开发了一个名为 SOUL 的基于第二阶修剪随机优化的遗忘框架，扩展了传统的一次性模型更新到动态的迭代遗忘过程，消除了影响数据对模型的影响，并在各种遗忘任务、模型和指标上 consistently outperforms 一阶方法，为大型语言模型的遗忘问题提供了可扩展且易于实施的解决方案。

SOUL：解锁第二阶优化对 LLM 取消学习的力量

SOUL: Unlocking the Power of Second-Order Optimization for LLM  Unlearning

Domain generalization (DG) seeks to learn robust models that generalize well
under unknown distribution shifts. As a critical aspect of DG, optimizer
selection has not been explored in depth. Currently, most DG methods follow the
widely used benchmark, DomainBed, and utilize Adam as the default optimizer for
all datasets. However, we reveal that Adam is not necessarily the optimal
choice for the majority of current DG methods and datasets. Based on the
perspective of loss landscape flatness, we propose a novel approach,
Flatness-Aware Minimization for Domain Generalization (FAD), which can
efficiently optimize both zeroth-order and first-order flatness simultaneously
for DG. We provide theoretical analyses of the FAD's out-of-distribution (OOD)
generalization error and convergence. Our experimental results demonstrate the
superiority of FAD on various DG datasets. Additionally, we confirm that FAD is
capable of discovering flatter optima in comparison to other zeroth-order and
first-order flatness-aware optimization methods.

基于损失平面平坦度的角度，我们提出了一种新颖的方法 Flatness-Aware Minimization for Domain Generalization（FAD），可以同时有效地优化零阶和一阶平坦度，从而改善领域泛化问题。我们通过理论分析和实验证明了 FAD 在各种领域泛化数据集上的优越性，并确认 FAD 能够发现比其他零阶和一阶平坦度感知优化方法更平坦的极小值点。

面向域一般化的平坦性感知最小化

Flatness-Aware Minimization for Domain Generalization

Despite a growing body of work at the intersection of deep learning and
formal languages, there has been relatively little systematic exploration of
transformer models for reasoning about typed lambda calculi. This is an
interesting area of inquiry for two reasons. First, typed lambda calculi are
the lingua franc of programming languages. A set of heuristics that relate
various typed lambda calculi to effective neural architectures would provide a
systematic method for mapping language features (e.g., polymorphism, subtyping,
inheritance, etc.) to architecture choices. Second, transformer models are
widely used in deep learning architectures applied to code, but the design and
hyperparameter space for them is large and relatively unexplored in programming
language applications. Therefore, we suggest a benchmark that allows us to
explore exactly this through perhaps the simplest and most fundamental property
of a programming language: the relationship between terms and types.
Consequently, we begin this inquiry of transformer architectures for typed
lambda calculi by exploring the effect of transformer warm-up and optimizer
selection in the task of type inference: i.e., predicting the types of lambda
calculus terms using only transformers. We find that the optimization landscape
is difficult even in this simple setting. One particular experimental finding
is that optimization by Adafactor converges much faster compared to the
optimization by Adam and RAdam. We conjecture that such different performance
of optimizers might be related to the difficulties of generalization over
formally generated dataset.

本文探讨了使用 transformer 模型对 typed lambda 演算进行类型推导的问题，并对优化器的选择以及 warm-up 的影响进行了实验，发现 Adafactor 比 Adam 和 RAdam 更有效。