In resource limited computing systems, sequence prediction models must
operate under tight constraints. Various models are available that cater to
prediction under these conditions that in some way focus on reducing the cost
of implementation. These resource constrained sequence prediction models, in
practice, exhibit a fundamental tradeoff between the cost of implementation and
the quality of its predictions. This fundamental tradeoff seems to be largely
unexplored for models for different tasks. Here we formulate the necessary
theory and an associated empirical procedure to explore this tradeoff space for
a particular family of machine learning models such as deep neural networks. We
anticipate that the knowledge of the behavior of this tradeoff may be
beneficial in understanding the theoretical and practical limits of creation
and deployment of models for resource constrained tasks.

在资源受限的计算系统中，序列预测模型必须在严格约束下运行。我们提出了一种理论和相关的实证方法，探索一种特定类型机器学习模型（如深度神经网络）的成本与预测质量之间的权衡空间，以增加对资源受限任务模型的理论和实践限制的认识。

程序阶段的模型复杂度

Model Complexity of Program Phases

Sequence prediction models can be learned from example sequences with a
variety of training algorithms. Maximum likelihood learning is simple and
efficient, yet can suffer from compounding error at test time. Reinforcement
learning such as policy gradient addresses the issue but can have prohibitively
poor exploration efficiency. A rich set of other algorithms such as RAML, SPG,
and data noising, have also been developed from different perspectives. This
paper establishes a formal connection between these algorithms. We present a
generalized entropy regularized policy optimization formulation, and show that
the apparently distinct algorithms can all be reformulated as special instances
of the framework, with the only difference being the configurations of a reward
function and a couple of hyperparameters. The unified interpretation offers a
systematic view of the varying properties of exploration and learning
efficiency. Besides, inspired from the framework, we present a new algorithm
that dynamically interpolates among the family of algorithms for scheduled
sequence model learning. Experiments on machine translation, text
summarization, and game imitation learning demonstrate the superiority of the
proposed algorithm.

采用统一的熵正则化策略优化框架，将不同算法统一为特殊实例，从而提供了统一的探索与学习效率的视角。此外，本文还提出了一种动态插值的算法，用于调度序列模型的学习，实验证明其优于传统算法。

MLE 和 RL 在序列预测中的联系

Connecting the Dots Between MLE and RL for Sequence Prediction

We study the problem of predicting a set or list of options under knapsack
constraint. The quality of such lists are evaluated by a submodular reward
function that measures both quality and diversity. Similar to DAgger (Ross et
al., 2010), by a reduction to online learning, we show how to adapt two
sequence prediction models to imitate greedy maximization under knapsack
constraint problems: CONSEQOPT (Dey et al., 2012) and SCP (Ross et al., 2013).
Experiments on extractive multi-document summarization show that our approach
outperforms existing state-of-the-art methods.

本文研究了在背包约束下预测选项集或列表的问题，并通过在线学习的降级模型，证明了如何使两个序列预测模型适应贪心的最大化背包约束问题，实验结果表明，我们的方法优于现有的最先进方法。