In computer vision, 2D convolution is arguably the most important operation
performed by a ConvNet. Unsurprisingly, it has been the focus of intense
software and hardware optimization and enjoys highly efficient implementations.
In this work, we ask an intriguing question: can we make a ConvNet work without
2D convolutions? Surprisingly, we find that the answer is yes -- we show that a
ConvNet consisting entirely of 1D convolutions can do just as well as 2D on
ImageNet classification. Specifically, we find that one key ingredient to a
high-performing 1D ConvNet is oriented 1D kernels: 1D kernels that are oriented
not just horizontally or vertically, but also at other angles. Our experiments
show that oriented 1D convolutions can not only replace 2D convolutions but
also augment existing architectures with large kernels, leading to improved
accuracy with minimal FLOPs increase. A key contribution of this work is a
highly-optimized custom CUDA implementation of oriented 1D kernels, specialized
to the depthwise convolution setting. Our benchmarks demonstrate that our
custom CUDA implementation almost perfectly realizes the theoretical advantage
of 1D convolution: it is faster than a native horizontal convolution for any
arbitrary angle. Code is available at
this https URL

利用定向的一维卷积核能够替代二维卷积，并且在 ImageNet 分类中表现出与二维卷积相当的准确性。

具有定向 1D 核的卷积网络

Convolutional Networks with Oriented 1D Kernels

Industrial control systems (ICSs) are widely used and vital to industry and
society. Their failure can have severe impact on both economics and human life.
Hence, these systems have become an attractive target for attacks, both
physical and cyber. A number of attack detection methods have been proposed,
however they are characterized by a low detection rate, a substantial false
positive rate, or are system specific. In this paper, we study an attack
detection method based on simple and lightweight neural networks, namely, 1D
convolutions and autoencoders. We apply these networks to both the time and
frequency domains of the collected data and discuss pros and cons of each
approach. We evaluate the suggested method on three popular public datasets and
achieve detection rates matching or exceeding previously published detection
results, while featuring small footprint, short training and detection times,
and generality. We also demonstrate the effectiveness of PCA, which, given
proper data preprocessing and feature selection, can provide high attack
detection scores in many settings. Finally, we study the proposed method's
robustness against adversarial attacks, that exploit inherent blind spots of
neural networks to evade detection while achieving their intended physical
effect. Our results show that the proposed method is robust to such evasion
attacks: in order to evade detection, the attacker is forced to sacrifice the
desired physical impact on the system. This finding suggests that neural
networks trained under the constraints of the laws of physics can be trusted
more than networks trained under more flexible conditions.

本文研究基于 1D 卷积和自编码器的简单轻量级神经网络攻击检测方法，将此方法应用于时间和频率域，评估结果表明，该方法能够匹配或超过先前的检测结果，同时具有小的足迹、短的训练和检测时间及普适性。此外，本文研究了所提方法对敌对攻击的鲁棒性，结果发现所提方法对敌对攻击具有强鲁棒性，可以受到更多信任。

使用轻量级神经网络和主成分分析在工业控制系统中进行高效的网络攻击检测

Efficient Cyber Attacks Detection in Industrial Control Systems Using  Lightweight Neural Networks and PCA

In this paper, we report state-of-the-art results on LibriSpeech among
end-to-end speech recognition models without any external training data. Our
model, Jasper, uses only 1D convolutions, batch normalization, ReLU, dropout,
and residual connections. To improve training, we further introduce a new
layer-wise optimizer called NovoGrad. Through experiments, we demonstrate that
the proposed deep architecture performs as well or better than more complex
choices. Our deepest Jasper variant uses 54 convolutional layers. With this
architecture, we achieve 2.95% WER using a beam-search decoder with an external
neural language model and 3.86% WER with a greedy decoder on LibriSpeech
test-clean. We also report competitive results on the Wall Street Journal and
the Hub5'00 conversational evaluation datasets.

本文提出一种仅使用 1D 卷积、批量归一化、ReLU、dropout 和残差连接的端到端语音识别模型 Jasper，结合新的层次优化器 NovoGrad，实现更好的训练和更高的准确率，最先进的 Jasper 变体使用 54 个卷积层，基于 LibriSpeech test-clean 数据集使用束搜索解码器和外部神经语言模型实现 2.95% WER 和贪心解码器实现 3.86% WER，并在 Wall Street Journal 和 Hub5'00 交际评估数据集上获得了有竞争力的结果。

Jasper: 一种端到端卷积神经声学模型

Jasper: An End-to-End Convolutional Neural Acoustic Model

Deep learning and convolutional neural networks (ConvNets) have been
successfully applied to most relevant tasks in the computer vision community.
However, these networks are computationally demanding and not suitable for
embedded devices where memory and time consumption are relevant.
In this paper, we propose DecomposeMe, a simple but effective technique to
learn features using 1D convolutions. The proposed architecture enables both
simplicity and filter sharing leading to increased learning capacity. A
comprehensive set of large-scale experiments on ImageNet and Places2
demonstrates the ability of our method to improve performance while
significantly reducing the number of parameters required. Notably, on Places2,
we obtain an improvement in relative top-1 classification accuracy of 7.7\%
with an architecture that requires 92% fewer parameters compared to VGG-B. The
proposed network is also demonstrated to generalize to other tasks by
converting existing networks.

本文提出了一种名为 DecomposeMe 的简单而有效的技术，使用 1D 卷积来学习特征，通过加强滤波器共享，以提高学习能力。作者使用 ImageNet 数据集和 Places2 数据集进行了一组实验，结果显示此技术可在显著减少需要的参数数量的同时提高网络性能，在 Places2 数据集上相对 top-1 分类精度提高了 7.7％且所需的参数数量比 VGG-B 少 92％，并可转换为其他任务的网络结构。