Reactive languages are dedicated to the programming of systems which interact
continuously and concurrently with their environment. Values take the form of
unbounded streams modeling the (discrete) passing of time or the sequence of
concurrent interactions. While conventional reactivity models recurrences
forward in time, we introduce a symmetric reactive construct enabling backward
recurrences. Constraints on the latter allow to make the implementation
practical. Machine Learning (ML) systems provide numerous motivations for all
of this: we demonstrate that reverse-mode automatic differentiation,
backpropagation, batch normalization, bidirectional recurrent neural networks,
training and reinforcement learning algorithms, are all naturally captured as
bidirectional reactive programs.

反应式语言是用于与环境连续并发交互的系统的编程。值采用无界流的形式，模拟时间的离散流逝或并发交互的序列。我们引入了一种对称的反应式构造，使得反向递归成为可能。对后者的约束使得实现变得实际。机器学习系统提供了许多动机：我们证明了反向模式的自动微分，反向传播，批量归一化，双向递归神经网络，训练和强化学习算法都可以自然地作为双向反应程序捕获。

双向响应式编程的机器学习

Bidirectional Reactive Programming for Machine Learning

Despite their ubiquity in NLP tasks, Long Short-Term Memory (LSTM) networks
suffer from computational inefficiencies caused by inherent unparallelizable
recurrences, which further aggravates as LSTMs require more parameters for
larger memory capacity. In this paper, we propose to apply low-rank matrix
factorization (MF) algorithms to different recurrences in LSTMs, and explore
the effectiveness on different NLP tasks and model components. We discover that
additive recurrence is more important than multiplicative recurrence, and
explain this by identifying meaningful correlations between matrix norms and
compression performance. We compare our approach across two settings: 1)
compressing core LSTM recurrences in language models, 2) compressing biLSTM
layers of ELMo evaluated in three downstream NLP tasks.

本文提出了一种将低秩矩阵分解算法应用于 LSTM 网络不同递归操作的方法，探讨了在不同自然语言处理任务和模型组成部分中的有效性，并发现加性递归比乘性递归更重要。两种设置下我们进行了比较：1）压缩语言模型中核心 LSTM 递归，2）压缩在三个下游自然语言处理任务中评估 ELMo 的 biLSTM 层。

低秩矩阵分解在 LSTM 模型压缩中的有效性研究

On the Effectiveness of Low-Rank Matrix Factorization for LSTM Model  Compression

This paper investigates modifying an existing neural network architecture for
static saliency prediction using two types of recurrences that integrate
information from the temporal domain. The first modification is the addition of
a ConvLSTM within the architecture, while the second is a conceptually simple
exponential moving average of an internal convolutional state. We use weights
pre-trained on the SALICON dataset and fine-tune our model on DHF1K. Our
results show that both modifications achieve state-of-the-art results and
produce similar saliency maps. Source code is available at
this https URL

本文探讨了利用两种类型的循环神经网络来改进现有的神经网络架构，以进行静态显著性预测。第一种修改是在架构内添加 ConvLSTM，而第二种修改是卷积状态的指数移动平均。实验结果在 SALICON 数据集上进行了权重预训练，并在 DHF1K 上进行了微调，结果表明两种修改均达到了最先进的效果，并产生了类似的显著性映射。可以在该 https URL 处获取源代码。