Conventional training methods for artificial neural network (ANN) models
never achieve zero error rate systematically for large data. A new training
method consists of three steps: first create an auxiliary data from
conventionally trained parameters which correspond exactly to a global minimum
for the loss function of the cloned data; second create a one-parameter
homotopy (hybrid) of the auxiliary data and the original data; and third train
the model for the hybrid data iteratively from the auxiliary data end of the
homotopy parameter to the original data end while maintaining the zero-error
training rate at every iteration. This continuationmethod is guaranteed to
converge numerically by a theorem which converts the ANN training problem into
a continuation problem for fixed points of a parameterized transformation in
the training parameter space to which the Uniform Contraction Mapping Theorem
from dynamical systems applies.

传统的人工神经网络训练方法无法系统地实现大数据的零错误率。一种新的训练方法包括三个步骤：首先从传统训练的参数创建一个辅助数据，其对应于克隆数据的损失函数的全局最小值；其次，创建辅助数据和原始数据的一个参数连续（混合）；第三，通过保持每次迭代的零错误训练率，从混合数据的辅助数据端到原始数据端迭代地训练模型。这种连续化方法通过将人工神经网络训练问题转换为训练参数空间中参数化变换的不动点的连续化问题，以动力系统中统一收缩映射定理来保证数值上的收敛。

人工神经网络的无误训练

Error-free Training for Artificial Neural Network

Learning to hash has been widely applied to approximate nearest neighbor
search for large-scale multimedia retrieval, due to its computation efficiency
and retrieval quality. Deep learning to hash, which improves retrieval quality
by end-to-end representation learning and hash encoding, has received
increasing attention recently. Subject to the ill-posed gradient difficulty in
the optimization with sign activations, existing deep learning to hash methods
need to first learn continuous representations and then generate binary hash
codes in a separated binarization step, which suffer from substantial loss of
retrieval quality. This work presents HashNet, a novel deep architecture for
deep learning to hash by continuation method with convergence guarantees, which
learns exactly binary hash codes from imbalanced similarity data. The key idea
is to attack the ill-posed gradient problem in optimizing deep networks with
non-smooth binary activations by continuation method, in which we begin from
learning an easier network with smoothed activation function and let it evolve
during the training, until it eventually goes back to being the original,
difficult to optimize, deep network with the sign activation function.
Comprehensive empirical evidence shows that HashNet can generate exactly binary
hash codes and yield state-of-the-art multimedia retrieval performance on
standard benchmarks.

HashNet 是一种基于连续化方法具有收敛保证的新型深度学习哈希结构，它从不平衡的相似性数据中学习准确的二进制哈希码，在标准测试中可产生最先进的多媒体检索性能。

HashNet: 基于连续方法的深度哈希学习

HashNet: Deep Learning to Hash by Continuation

A simple procedure to map two probability measures in $\mathbb{R}^d$ is the
so-called \emph{Knothe-Rosenblatt rearrangement}, which consists in rearranging
monotonically the marginal distributions of the last coordinate, and then the
conditional distributions, iteratively. We show that this mapping is the limit
of solutions to a class of Monge-Kantorovich mass transportation problems with
quadratic costs, with the weights of the coordinates asymptotically dominating
one another. This enables us to design a continuation method for numerically
solving the optimal transport problem.

本文介绍了一种基于 Knothe-Rosenblatt 置换的转换方法，该方法可用于解决具有二次代价的 Monge-Kantorovich 质量输运问题，详细介绍了优化输运问题的数值解法。