Many machine translation toolkits make use of a data preparation step wherein
raw data is transformed into a tensor format that can be used directly by the
trainer. This preparation step is increasingly at odds with modern research and
development practices because this process produces a static, unchangeable
version of the training data, making common training-time needs difficult
(e.g., subword sampling), time-consuming (preprocessing with large data can
take days), expensive (e.g., disk space), and cumbersome (managing experiment
combinatorics). We propose an alternative approach that separates the
generation of data from the consumption of that data. In this approach, there
is no separate pre-processing step; data generation produces an infinite stream
of permutations of the raw training data, which the trainer tensorizes and
batches as it is consumed. Additionally, this data stream can be manipulated by
a set of user-definable operators that provide on-the-fly modifications, such
as data normalization, augmentation or filtering. We release an open-source
toolkit, SOTASTREAM, that implements this approach:
this https URL We show that it cuts training time,
adds flexibility, reduces experiment management complexity, and reduces disk
space, all without affecting the accuracy of the trained models.

许多机器翻译工具包利用数据准备步骤将原始数据转换为可以直接由训练器使用的张量格式。我们提出了一种替代方法，将数据的生成与使用分离，通过生成原始训练数据的无限排列流，并通过一组可自定义的操作符进行实时修改，从而提高训练效率、灵活性和模型的准确性，同时降低了实验管理复杂性和磁盘空间需求。

SOTASTREAM: 机器翻译训练的流式方法

SOTASTREAM: A Streaming Approach to Machine Translation Training

Convolutional neural networks are now seeing widespread use in a variety of
fields, including image classification, facial and object recognition, medical
imaging analysis, and many more. In addition, there are applications such as
physics-informed simulators in which accurate forecasts in real time with a
minimal lag are required. The present neural network designs include millions
of parameters, which makes it difficult to install such complex models on
devices that have limited memory. Compression techniques might be able to
resolve these issues by decreasing the size of CNN models that are created by
reducing the number of parameters that contribute to the complexity of the
models. We propose a compressed tensor format of convolutional layer, a priori,
before the training of the neural network. 3-way kernels or 2-way kernels in
convolutional layers are replaced by one-way fiters. The overfitting phenomena
will be reduced also. The time needed to make predictions or time required for
training using the original Convolutional Neural Networks model would be cut
significantly if there were fewer parameters to deal with. In this paper we
present a method of a priori compressing convolutional neural networks for
finite element (FE) predictions of physical data. Afterwards we validate our a
priori compressed models on physical data from a FE model solving a 2D wave
equation. We show that the proposed convolutinal compression technique achieves
equivalent performance as classical convolutional layers with fewer trainable
parameters and lower memory footprint.

本文提出使用压缩张量格式的卷积层、使用单向滤波器替代卷积层的三向 / 二向滤波器等方式压缩卷积神经网络模型，减少参数数量，从而降低内存占用，在预测时间和训练时间上表现良好。同时，文章将此技术应用于预测有限元物理数据。