Fine-grained classification is a particular case of a classification problem,
aiming to classify objects that share the visual appearance and can only be
distinguished by subtle differences. Fine-grained classification models are
often deployed to determine animal species or individuals in automated animal
monitoring systems. Precise visual explanations of the model's decision are
crucial to analyze systematic errors. Attention- or gradient-based methods are
commonly used to identify regions in the image that contribute the most to the
classification decision. These methods deliver either too coarse or too noisy
explanations, unsuitable for identifying subtle visual differences reliably.
However, perturbation-based methods can precisely identify pixels causally
responsible for the classification result. Fill-in of the dropout (FIDO)
algorithm is one of those methods. It utilizes the concrete dropout (CD) to
sample a set of attribution masks and updates the sampling parameters based on
the output of the classification model. A known problem of the algorithm is a
high variance in the gradient estimates, which the authors have mitigated until
now by mini-batch updates of the sampling parameters. This paper presents a
solution to circumvent these computational instabilities by simplifying the CD
sampling and reducing reliance on large mini-batch sizes. First, it allows
estimating the parameters with smaller mini-batch sizes without losing the
quality of the estimates but with a reduced computational effort. Furthermore,
our solution produces finer and more coherent attribution masks. Finally, we
use the resulting attribution masks to improve the classification performance
of a trained model without additional fine-tuning of the model.

通过简化采样和减少对大型小批量大小的依赖，本研究提出了一种解决方案来绕过计算不稳定性，产生更细致、更连贯的归因掩码，并利用所得到的归因掩码改善已训练模型的分类性能，无需进行额外的微调。

简化的具体丢弃 -- 改进了对细粒度分类的属性掩码生成

Simplified Concrete Dropout -- Improving the Generation of Attribution  Masks for Fine-grained Classification

Skeleton based action recognition distinguishes human actions using the
trajectories of skeleton joints, which provide a very good representation for
describing actions. Considering that recurrent neural networks (RNNs) with Long
Short-Term Memory (LSTM) can learn feature representations and model long-term
temporal dependencies automatically, we propose an end-to-end fully connected
deep LSTM network for skeleton based action recognition. Inspired by the
observation that the co-occurrences of the joints intrinsically characterize
human actions, we take the skeleton as the input at each time slot and
introduce a novel regularization scheme to learn the co-occurrence features of
skeleton joints. To train the deep LSTM network effectively, we propose a new
dropout algorithm which simultaneously operates on the gates, cells, and output
responses of the LSTM neurons. Experimental results on three human action
recognition datasets consistently demonstrate the effectiveness of the proposed
model.

提出了一种基于骨架的行为识别模型，使用递归神经网络（RNNs）与长短时记忆（LSTM）建立深度学习网络，学习并自动建模骨架关节之间的协同特征，在三个人类行为识别数据集上具有良好的效果。

基于正则化深层 LSTM 网络的骨架动作识别的共现特征学习

Co-occurrence Feature Learning for Skeleton based Action Recognition  using Regularized Deep LSTM Networks

Catastrophic forgetting is a problem faced by many machine learning models
and algorithms. When trained on one task, then trained on a second task, many
machine learning models "forget" how to perform the first task. This is widely
believed to be a serious problem for neural networks. Here, we investigate the
extent to which the catastrophic forgetting problem occurs for modern neural
networks, comparing both established and recent gradient-based training
algorithms and activation functions. We also examine the effect of the
relationship between the first task and the second task on catastrophic
forgetting. We find that it is always best to train using the dropout
algorithm--the dropout algorithm is consistently best at adapting to the new
task, remembering the old task, and has the best tradeoff curve between these
two extremes. We find that different tasks and relationships between tasks
result in very different rankings of activation function performance. This
suggests the choice of activation function should always be cross-validated.

探究现代神经网络机器学习模型在不同任务训练后的 “灾难性遗忘” 问题，发现以 dropout 算法为代表的梯度训练算法可以最好地适应新任务并记住旧任务，而不同任务间的关系会显著影响激活函数表现，建议激活函数选择交叉验证。