Variational autoencoders (VAE) are directed generative models that learn
factorial latent variables. As noted by Burda et al. (2015), these models
exhibit the problem of factor over-pruning where a significant number of
stochastic factors fail to learn anything and become inactive. This can limit
their modeling power and their ability to learn diverse and meaningful latent
representations. In this paper, we evaluate several methods to address this
problem and propose a more effective model-based approach called the epitomic
variational autoencoder (eVAE). The so-called epitomes of this model are groups
of mutually exclusive latent factors that compete to explain the data. This
approach helps prevent inactive units since each group is pressured to explain
the data. We compare the approaches with qualitative and quantitative results
on MNIST and TFD datasets. Our results show that eVAE makes efficient use of
model capacity and generalizes better than VAE.

本研究比较了多种方法来解决变分自编码器中随机因素无法学习和失活等问题，并提出了一种更有效的基于模型的方法，称为最佳变分自编码器，该方法有助于防止单元失活，并比 VAE 更好地使用了模型容量，产生更好的泛化性能。

解决变分自编码器中的过度剪枝问题

Tackling Over-pruning in Variational Autoencoders

Recurrent neural networks (RNNs) have been used extensively and with
increasing success to model various types of sequential data. Much of this
progress has been achieved through devising recurrent units and architectures
with the flexibility to capture complex statistics in the data, such as long
range dependency or localized attention phenomena. However, while many
sequential data (such as video, speech or language) can have highly variable
information flow, most recurrent models still consume input features at a
constant rate and perform a constant number of computations per time step,
which can be detrimental to both speed and model capacity. In this paper, we
explore a modification to existing recurrent units which allows them to learn
to vary the amount of computation they perform at each step, without prior
knowledge of the sequence's time structure. We show experimentally that not
only do our models require fewer operations, they also lead to better
performance overall on evaluation tasks.

本研究提出了一种改进的循环单元，使其能够在学习时不需要事先了解序列的时间结构就能够灵活地调整每个时间步骤的计算量，从而提高模型的容量和性能，实验证明该方法能够在评估任务中取得更好的性能表现并减少计算量。

递归神经网络中的变量计算

Variable Computation in Recurrent Neural Networks

This paper investigates neural character-based morphological tagging for
languages with complex morphology and large tag sets. We systematically explore
a variety of neural architectures (DNN, CNN, CNNHighway, LSTM, BLSTM) to obtain
character-based word vectors combined with bidirectional LSTMs to model
across-word context in an end-to-end setting. We explore supplementary use of
word-based vectors trained on large amounts of unlabeled data. Our experiments
for morphological tagging suggest that for "simple" model configurations, the
choice of the network architecture (CNN vs. CNNHighway vs. LSTM vs. BLSTM) or
the augmentation with pre-trained word embeddings can be important and clearly
impact the accuracy. Increasing the model capacity by adding depth, for
example, and carefully optimizing the neural networks can lead to substantial
improvements, and the differences in accuracy (but not training time) become
much smaller or even negligible. Overall, our best morphological taggers for
German and Czech outperform the best results reported in the literature by a
large margin.

本文探讨了适用于复杂形态和大标注集语言的神经字符基础形态标注，结合双向 LSTMs 建模跨单词上下文，发现网络架构和预训练词嵌入向量在‘简单’模型配置的情况下对准确度产生重要影响，通过增加深度优化神经网络可显著提高标注器准确度，最终德语和捷克语的最佳形态标注器性能显著优于文献中最好结果。