Current language generation models suffer from issues such as repetition,
incoherence, and hallucinations. An often-repeated hypothesis is that this
brittleness of generation models is caused by the training and the generation
procedure mismatch, also referred to as exposure bias. In this paper, we verify
this hypothesis by analyzing exposure bias from an imitation learning
perspective. We show that exposure bias leads to an accumulation of errors,
analyze why perplexity fails to capture this accumulation, and empirically show
that this accumulation results in poor generation quality. Source code to
reproduce these experiments is available at
this https URL

本文探讨语言生成模型存在的问题，阐述了模型脆弱性产生的原因，并从模仿学习的角度分析暴露偏差。作者证实了该假设，通过分析误差积累的原因和导致质量低下的积累。

曝光偏差的重要性：语言生成中误差累积的模仿学习视角

Why Exposure Bias Matters: An Imitation Learning Perspective of Error Accumulation in Language Generation

In this paper, we investigate the sample size requirement for a general class
of nuclear norm minimization methods for higher order tensor completion. We
introduce a class of tensor norms by allowing for different levels of
coherence, which allows us to leverage the incoherence of a tensor. In
particular, we show that a $k$th order tensor of rank $r$ and dimension
$d\times\cdots\times d$ can be recovered perfectly from as few as
$O((r^{(k-1)/2}d^{3/2}+r^{k-1}d)(\log(d))^2)$ uniformly sampled entries through
an appropriate incoherent nuclear norm minimization. Our results demonstrate
some key differences between completing a matrix and a higher order tensor:
They not only point to potential room for improvement over the usual nuclear
norm minimization but also highlight the importance of explicitly accounting
for incoherence, when dealing with higher order tensors.

本文研究了高阶张量完成的核范数最小化算法中的样本数量要求，并通过引入一类张量范数，通过利用张量的不相干性，证明了一个阶数为 k，秩为 r，尺寸为 d x ... x d 的张量可以通过适当的不相干核范数最小化算法从仅采样 O（（r ^（k-1）/ 2} d ^ {3/2} + r ^ {k-1} d）（log（d））^ 2）个条目中完美恢复。研究结果不仅指出了当前核范数最小化算法的潜在改进空间，而且还强调了在处理高阶张量时明确考虑不相干性的重要性。

非相干张量范数及其在高阶张量补全中的应用

Incoherent Tensor Norms and Their Applications in Higher Order Tensor  Completion

This paper investigates the problem of determining a binary-valued function
through a sequence of strategically selected queries. The focus is an algorithm
called Generalized Binary Search (GBS). GBS is a well-known greedy algorithm
for determining a binary-valued function through a sequence of strategically
selected queries. At each step, a query is selected that most evenly splits the
hypotheses under consideration into two disjoint subsets, a natural
generalization of the idea underlying classic binary search. This paper
develops novel incoherence and geometric conditions under which GBS achieves
the information-theoretically optimal query complexity; i.e., given a
collection of N hypotheses, GBS terminates with the correct function after no
more than a constant times log N queries. Furthermore, a noise-tolerant version
of GBS is developed that also achieves the optimal query complexity. These
results are applied to learning halfspaces, a problem arising routinely in
image processing and machine learning.

本文研究了一个叫做广义二分搜索的算法，通过一系列策略性的查询确定二值函数，提出了新的不连贯性和几何条件，证明其信息理论最优查询复杂度。此外，该算法还应用于学习半空间问题。