Deep models are dominating the artificial intelligence (AI) industry since
the ImageNet challenge in 2012. The size of deep models is increasing ever
since, which brings new challenges to this field with applications in cell
phones, personal computers, autonomous cars, and wireless base stations. Here
we list a set of problems, ranging from training, inference, generalization
bound, and optimization with some formalism to communicate these challenges
with mathematicians, statisticians, and theoretical computer scientists. This
is a subjective view of the research questions in deep learning that benefits
the tech industry in long run.

深度学习在 AI 工业中占据主导地位，但其规模不断扩大也带来了一些从训练到推理、泛化到优化等方面的新挑战，本文总结了这些挑战并提出了一些解决方案，为数学家、统计学家和理论计算机科学家讨论该领域的研究问题提供了一些主观观点，有助于其在技术产业中的长期应用。

深度学习中的数学挑战

Mathematical Challenges in Deep Learning

The right to be forgotten has been legislated in many countries, but its
enforcement in the AI industry would cause unbearable costs. When single data
deletion requests come, companies may need to delete the whole models learned
with massive resources. Existing works propose methods to remove knowledge
learned from data for explicitly parameterized models, which however are not
appliable to the sampling-based Bayesian inference, i.e., Markov chain Monte
Carlo (MCMC), as MCMC can only infer implicit distributions. In this paper, we
propose the first machine unlearning algorithm for MCMC. We first convert the
MCMC unlearning problem into an explicit optimization problem. Based on this
problem conversion, an {\it MCMC influence function} is designed to provably
characterize the learned knowledge from data, which then delivers the MCMC
unlearning algorithm. Theoretical analysis shows that MCMC unlearning would not
compromise the generalizability of the MCMC models. Experiments on Gaussian
mixture models and Bayesian neural networks confirm the effectiveness of the
proposed algorithm. The code is available at
https://github.com/fshp971/mcmc-unlearning.

研究讨论在人工智能领域中实施遗忘权利将会引起难以承受的成本问题，该论文提出了用于 Markov chain Monte Carlo （简称 MCMC）的机器遗忘算法，第一次将 MCMC 反向推导为明确的最优化问题，通过 MCMC 引导函数确定遗忘的知识来达到遗忘的目的，并证明该算法不会损害 MCMC 模型的泛化能力。