We present Pathway, a new unified data processing framework that can run
workloads on both bounded and unbounded data streams. The framework was created
with the original motivation of resolving challenges faced when analyzing and
processing data from the physical economy, including streams of data generated
by IoT and enterprise systems. These required rapid reaction while calling for
the application of advanced computation paradigms (machinelearning-powered
analytics, contextual analysis, and other elements of complex event
processing). Pathway is equipped with a Table API tailored for Python and
Python/SQL workflows, and is powered by a distributed incremental dataflow in
Rust. We describe the system and present benchmarking results which demonstrate
its capabilities in both batch and streaming contexts, where it is able to
surpass state-of-the-art industry frameworks in both scenarios. We also discuss
streaming use cases handled by Pathway which cannot be easily resolved with
state-of-the-art industry frameworks, such as streaming iterative graph
algorithms (PageRank, etc.).

我们提出了 Pathway，这是一个新的统一数据处理框架，可以在有界和无界数据流上运行工作负载。该框架旨在解决分析和处理物理经济数据时面临的挑战，包括物联网和企业系统生成的数据流。我们描述了该系统，并展示了其在批处理和流处理环境中的能力，超过了业界最先进的框架。我们还讨论了 Pathway 处理的流处理用例，这些用例无法轻松解决使用业界最先进的框架，例如流迭代图算法（PageRank 等）。

Pathway：一种快速灵活的统一流数据处理框架，用于分析和机器学习应用

Pathway: a fast and flexible unified stream data processing framework  for analytical and Machine Learning applications

We study the application of active learning techniques to the translation of
unbounded data streams via interactive neural machine translation. The main
idea is to select, from an unbounded stream of source sentences, those worth to
be supervised by a human agent. The user will interactively translate those
samples. Once validated, these data is useful for adapting the neural machine
translation model.
We propose two novel methods for selecting the samples to be validated. We
exploit the information from the attention mechanism of a neural machine
translation system. Our experiments show that the inclusion of active learning
techniques into this pipeline allows to reduce the effort required during the
process, while increasing the quality of the translation system. Moreover, it
enables to balance the human effort required for achieving a certain
translation quality. Moreover, our neural system outperforms classical
approaches by a large margin.

本文探讨了使用主动学习技术对无限数据流进行交互式神经机器翻译的应用，并提出了两种选择样本的新方法。我们的实验表明，将主动学习技术集成到翻译系统中可在增加翻译质量的同时降低所需的人力工作量，并使人力工作量达到一定的翻译质量平衡。此外，我们的神经系统表现明显优于传统方法。