Deep neural network autoencoders are routinely used computationally for model
reduction. They allow recognizing the intrinsic dimension of data that lie in a
$k$-dimensional subset $K$ of an input Euclidean space $\R^n$. The underlying
idea is to obtain both an encoding layer that maps $\R^n$ into $\R^k$ (called
the bottleneck layer or the space of latent variables) and a decoding layer
that maps $\R^k$ back into $\R^n$, in such a way that the input data from the
set $K$ is recovered when composing the two maps. This is achieved by adjusting
parameters (weights) in the network to minimize the discrepancy between the
input and the reconstructed output. Since neural networks (with continuous
activation functions) compute continuous maps, the existence of a network that
achieves perfect reconstruction would imply that $K$ is homeomorphic to a
$k$-dimensional subset of $\R^k$, so clearly there are topological obstructions
to finding such a network. On the other hand, in practice the technique is
found to ``work'' well, which leads one to ask if there is a way to explain
this effectiveness. We show that, up to small errors, indeed the method is
guaranteed to work. This is done by appealing to certain facts from
differential geometry. A computational example is also included to illustrate
the ideas.

深度神经网络自编码器常用于模型简化，在此研究中通过调整网络参数来最小化输入与重构输出之间的差异，以达到恢复输入数据的效果，并证明了该方法的有效性。

自编码器为何有效？

Why do autoencoders work?

This paper presents an unsupervised approach for writer retrieval based on
clustering SIFT descriptors detected at keypoint locations resulting in
pseudo-cluster labels. With those cluster labels, a residual network followed
by our proposed NetRVLAD, an encoding layer with reduced complexity compared to
NetVLAD, is trained on 32x32 patches at keypoint locations. Additionally, we
suggest a graph-based reranking algorithm called SGR to exploit similarities of
the page embeddings to boost the retrieval performance. Our approach is
evaluated on two historical datasets (Historical-WI and HisIR19). We include an
evaluation of different backbones and NetRVLAD. It competes with related work
on historical datasets without using explicit encodings. We set a new
State-of-the-art on both datasets by applying our reranking scheme and show
that our approach achieves comparable performance on a modern dataset as well.

本文提出了一种基于无监督方法的作者检索方法，其采用 SIFT 描述符进行聚类，并通过 NetRVLAD 进行编码，同时提出了一种基于图的重新排序算法（SGR）以提高检索性能，该方法在两个历史数据集上表现出色。

使用 NetRVLAD 和图相似度重新排名的无监督作家检索

Unsupervised Writer Retrieval using NetRVLAD and Graph Similarity  Reranking

In this paper, we explore the encoding/pooling layer and loss function in the
end-to-end speaker and language recognition system. First, a unified and
interpretable end-to-end system for both speaker and language recognition is
developed. It accepts variable-length input and produces an utterance level
result. In the end-to-end system, the encoding layer plays a role in
aggregating the variable-length input sequence into an utterance level
representation. Besides the basic temporal average pooling, we introduce a
self-attentive pooling layer and a learnable dictionary encoding layer to get
the utterance level representation. In terms of loss function for open-set
speaker verification, to get more discriminative speaker embedding, center loss
and angular softmax loss is introduced in the end-to-end system. Experimental
results on Voxceleb and NIST LRE 07 datasets show that the performance of
end-to-end learning system could be significantly improved by the proposed
encoding layer and loss function.

本研究探索了编码层、池化层和损失函数在端到端语音说话人和语言识别系统中的作用，开发了一个用于语音说话人和语言识别系统的端到端统一的、可解释的系统，并对编码层和损失函数进行了改进，实验结果表明改进后的端到端学习系统显著提高了性能。

探索端到端说话人和语言识别系统中的编码层和损失函数

Exploring the Encoding Layer and Loss Function in End-to-End Speaker and  Language Recognition System

We propose a Deep Texture Encoding Network (Deep-TEN) with a novel Encoding
Layer integrated on top of convolutional layers, which ports the entire
dictionary learning and encoding pipeline into a single model. Current methods
build from distinct components, using standard encoders with separate
off-the-shelf features such as SIFT descriptors or pre-trained CNN features for
material recognition. Our new approach provides an end-to-end learning
framework, where the inherent visual vocabularies are learned directly from the
loss function. The features, dictionaries and the encoding representation for
the classifier are all learned simultaneously. The representation is orderless
and therefore is particularly useful for material and texture recognition. The
Encoding Layer generalizes robust residual encoders such as VLAD and Fisher
Vectors, and has the property of discarding domain specific information which
makes the learned convolutional features easier to transfer. Additionally,
joint training using multiple datasets of varied sizes and class labels is
supported resulting in increased recognition performance. The experimental
results show superior performance as compared to state-of-the-art methods using
gold-standard databases such as MINC-2500, Flickr Material Database,
KTH-TIPS-2b, and two recent databases 4D-Light-Field-Material and GTOS. The
source code for the complete system are publicly available.

本文提出了一种基于深度学习的纹理编码网络 ——Deep-TEN，该网络集成了一种编码层，并通过端到端的学习框架直接学习词汇表和编码，支持多数据集联合训练，具有很好的纹理和物质识别性能。