Nonnegative tensor factorization (NTF) has become an important tool for
feature extraction and part-based representation with preserved intrinsic
structure information from nonnegative high-order data. However, the original
NTF methods utilize Euclidean or Kullback-Leibler divergence as the loss
function which treats each feature equally leading to the neglect of the
side-information of features. To utilize correlation information of features
and manifold information of samples, we introduce Wasserstein manifold
nonnegative tensor factorization (WMNTF), which minimizes the Wasserstein
distance between the distribution of input tensorial data and the distribution
of reconstruction. Although some researches about Wasserstein distance have
been proposed in nonnegative matrix factorization (NMF), they ignore the
spatial structure information of higher-order data. We use Wasserstein distance
(a.k.a Earth Mover's distance or Optimal Transport distance) as a metric and
add a graph regularizer to a latent factor. Experimental results demonstrate
the effectiveness of the proposed method compared with other NMF and NTF
methods.

利用 Wasserstein 距离作为度量，并将图形正则化器添加到潜在因子中，我们引入了 Wasserstein 流形非负张量分解（WMNTF），以利用特征的相关信息和样本的流形信息。实验证明，与其他 NMF 和 NTF 方法相比，所提方法的有效性。

具峰值非负张量分解的 Wasserstein 流形正则化

Wasserstein Nonnegative Tensor Factorization with Manifold  Regularization

Neural network pruning is an essential approach for reducing the
computational complexity of deep models so that they can be well deployed on
resource-limited devices. Compared with conventional methods, the recently
developed dynamic pruning methods determine redundant filters variant to each
input instance which achieves higher acceleration. Most of the existing methods
discover effective sub-networks for each instance independently and do not
utilize the relationship between different inputs. To maximally excavate
redundancy in the given network architecture, this paper proposes a new
paradigm that dynamically removes redundant filters by embedding the manifold
information of all instances into the space of pruned networks (dubbed as
ManiDP). We first investigate the recognition complexity and feature similarity
between images in the training set. Then, the manifold relationship between
instances and the pruned sub-networks will be aligned in the training
procedure. The effectiveness of the proposed method is verified on several
benchmarks, which shows better performance in terms of both accuracy and
computational cost compared to the state-of-the-art methods. For example, our
method can reduce 55.3% FLOPs of ResNet-34 with only 0.57% top-1 accuracy
degradation on ImageNet.

本论文提出了一种新的范式，通过将所有实例的流形信息嵌入到裁剪网络的空间中 (称为 ManiDP)，以动态地去除冗余的过滤器以最大化挖掘给定网络架构中的冗余。在几个基准测试中验证了所提出的方法的有效性，在精度和计算成本方面显示出与现有技术方法相比更好的性能可将 ResNet-34 的 FLOP 降低 55.3％，且仅仅减少 0.57％的 Top-1 精度，ImageNet。

流形正则化动态网络剪枝

Manifold Regularized Dynamic Network Pruning

Adversarial training is a popular defense strategy against attack threat
models with bounded Lp norms. However, it often degrades the model performance
on normal images and the defense does not generalize well to novel attacks.
Given the success of deep generative models such as GANs and VAEs in
characterizing the underlying manifold of images, we investigate whether or not
the aforementioned problems can be remedied by exploiting the underlying
manifold information. To this end, we construct an "On-Manifold ImageNet"
(OM-ImageNet) dataset by projecting the ImageNet samples onto the manifold
learned by StyleGSN. For this dataset, the underlying manifold information is
exact. Using OM-ImageNet, we first show that adversarial training in the latent
space of images improves both standard accuracy and robustness to on-manifold
attacks. However, since no out-of-manifold perturbations are realized, the
defense can be broken by Lp adversarial attacks. We further propose Dual
Manifold Adversarial Training (DMAT) where adversarial perturbations in both
latent and image spaces are used in robustifying the model. Our DMAT improves
performance on normal images, and achieves comparable robustness to the
standard adversarial training against Lp attacks. In addition, we observe that
models defended by DMAT achieve improved robustness against novel attacks which
manipulate images by global color shifts or various types of image filtering.
Interestingly, similar improvements are also achieved when the defended models
are tested on out-of-manifold natural images. These results demonstrate the
potential benefits of using manifold information in enhancing robustness of
deep learning models against various types of novel adversarial attacks.

通过在图像的潜在空间中对对抗样本进行对抗训练以及利用生成模型中学习到的流形信息进行双流形对抗训练，可以大大提高深度学习模型的鲁棒性，从而有效地应对多种新颖的对抗攻击。