We study the problem of estimating mixtures of Gaussians under the constraint
of differential privacy (DP). Our main result is that $\tilde{O}(k^2 d^4
\log(1/\delta) / \alpha^2 \varepsilon)$ samples are sufficient to estimate a
mixture of $k$ Gaussians up to total variation distance $\alpha$ while
satisfying $(\varepsilon, \delta)$-DP. This is the first finite sample
complexity upper bound for the problem that does not make any structural
assumptions on the GMMs.
To solve the problem, we devise a new framework which may be useful for other
tasks. On a high level, we show that if a class of distributions (such as
Gaussians) is (1) list decodable and (2) admits a "locally small'' cover
[BKSW19] with respect to total variation distance, then the class of its
mixtures is privately learnable. The proof circumvents a known barrier
indicating that, unlike Gaussians, GMMs do not admit a locally small cover
[AAL21].

在满足差分隐私的约束下，研究了估计混合高斯模型问题。通过使用新的框架，证明了高斯模型类的混合模型是可私密学习的，得到了估计混合高斯模型所需的样本数量的有界性，且不对 GMMs 作出任何结构性假设。

用多项式数量的样本来隐私学习高斯混合模型

Mixtures of Gaussians are Privately Learnable with a Polynomial Number  of Samples

In-context learning is a surprising and important phenomenon that emerged
when modern language models were scaled to billions of learned parameters.
Without modifying a large language model's weights, it can be tuned to perform
various downstream natural language tasks simply by including concatenated
training examples of these tasks in its input. Though disruptive for many
practical applications of large language models, this emergent learning
paradigm is not well understood from a theoretical perspective. In this paper,
we propose a first-of-its-kind PAC based framework for in-context learnability,
and use it to provide the first finite sample complexity results for the
in-context learning setup. Our framework includes an initial pretraining phase,
which fits a function to the pretraining distribution, and then a second
in-context learning phase, which keeps this function constant and concatenates
training examples of the downstream task in its input. We use our framework in
order to prove that, under mild assumptions, when the pretraining distribution
is a mixture of latent tasks (a model often considered for natural language
pretraining), these tasks can be efficiently learned via in-context learning,
even though the model's weights are unchanged and the input significantly
diverges from the pretraining distribution. Our theoretical analysis reveals
that in this setting, in-context learning is more about identifying the task
than about learning it, a result which is in line with a series of recent
empirical findings. We hope that the in-context learnability framework
presented in this paper will facilitate future progress towards a deeper
understanding of this important new learning paradigm.

本研究提出了一个基于 PAC 理论的框架来探究上下文学习及其可学性，发现在语言模型的参数保持不变的情况下，通过将下游任务的训练示例包含在其输入中，可以调整模型以执行各种下游自然语言处理任务，预训练分布是潜在任务的混合时，这些任务可以通过上下文学习有效地学习，这种学习更多地是关于识别任务而不是学习任务，并希望这一研究框架为深入理解上下文学习的新学习范式打下基础。