There has been a lot of recent interest in trying to characterize the error
surface of deep models. This stems from a long standing question. Given that
deep networks are highly nonlinear systems optimized by local gradient methods,
why do they not seem to be affected by bad local minima? It is widely believed
that training of deep models using gradient methods works so well because the
error surface either has no local minima, or if they exist they need to be
close in value to the global minimum. It is known that such results hold under
very strong assumptions which are not satisfied by real models. In this paper
we present examples showing that for such theorem to be true additional
assumptions on the data, initialization schemes and/or the model classes have
to be made. We look at the particular case of finite size datasets. We
demonstrate that in this scenario one can construct counter-examples (datasets
or initialization schemes) when the network does become susceptible to bad
local minima over the weight space.

本文主要探讨了对于深度模型的错误表面进行特征化的兴趣，揭示在某些条件下，深度模型的局部最小值会影响模型训练的结果，需要额外的数据前提、初始化方案和 / 或模型类来支持全局最佳解的优化。