This paper tackles the problem of minimizing the prediction loss of the
optimal solution (PLS) of the MILP with given data, which is one of the inverse
optimization problems. While existing methods can approximately solve this
problem, their implementation in the high-dimensional case to minimize the PLS
is computationally expensive because they are inefficient in reducing the
prediction loss of weights (PLW). We propose a fast algorithm for minimizing
the PLS of MILP. To demonstrate this property, we attribute the problem of
minimizing the PLS to that of minimizing the suboptimality loss (SL), which is
convex. If the PLS does not vanish, we can adapt the SL to have the estimated
loss (SPO loss) with a positive lower bound, which enables us to evaluate the
PLW. Consequently, we prove that the proposed algorithm can effectively reduce
the PLW and achieve the minimum value of PLS. Our numerical experiments
demonstrated that our algorithm successfully achieved the minimum PLS. Compared
to existing methods, our algorithm exhibited a smaller dimensionality effect
and minimized the PLS in less than 1/7 the number of iterations. Especially in
high dimensions, our algorithm significantly improved the PLS by more than two
orders of magnitude compared to existing algorithms.

通过将最小化预测损失的问题归结为最小化次优性损失的问题，我们提出了一种快速算法来有效地减少权重的预测损失并实现最小化的预测损失值。

一种快速算法以最小化多项式整数线性规划逆优化问题的预测损失

A fast algorithm to minimize prediction loss of the optimal solution in  inverse optimization problem of MILP

The vulnerability of deep neural networks (DNNs) to adversarial examples has
attracted great attention in the machine learning community. The problem is
related to non-flatness and non-smoothness of normally obtained loss
landscapes. Training augmented with adversarial examples (a.k.a., adversarial
training) is considered as an effective remedy. In this paper, we highlight
that some collaborative examples, nearly perceptually indistinguishable from
both adversarial and benign examples yet show extremely lower prediction loss,
can be utilized to enhance adversarial training. A novel method is therefore
proposed to achieve new state-of-the-arts in adversarial robustness. Code:
this https URL

本文提出了一种新的方法，利用协同示例来增强对抗性训练，从而在对抗性鲁棒性方面实现了新的最高水平。

对抗性鲁棒性的压缩训练

Squeeze Training for Adversarial Robustness

Self-supervised approaches for speech representation learning are challenged
by three unique problems: (1) there are multiple sound units in each input
utterance, (2) there is no lexicon of input sound units during the pre-training
phase, and (3) sound units have variable lengths with no explicit segmentation.
To deal with these three problems, we propose the Hidden-Unit BERT (HuBERT)
approach for self-supervised speech representation learning, which utilizes an
offline clustering step to provide aligned target labels for a BERT-like
prediction loss. A key ingredient of our approach is applying the prediction
loss over the masked regions only, which forces the model to learn a combined
acoustic and language model over the continuous inputs. HuBERT relies primarily
on the consistency of the unsupervised clustering step rather than the
intrinsic quality of the assigned cluster labels. Starting with a simple
k-means teacher of 100 clusters, and using two iterations of clustering, the
HuBERT model either matches or improves upon the state-of-the-art wav2vec 2.0
performance on the Librispeech (960h) and Libri-light (60,000h) benchmarks with
10min, 1h, 10h, 100h, and 960h fine-tuning subsets. Using a 1B parameter model,
HuBERT shows up to 19% and 13% relative WER reduction on the more challenging
dev-other and test-other evaluation subsets.

本研究提出了一个利用聚类和预测损失的自监督学习方法 HuBERT，来解决语音表示学习中存在的多个输入单元、无法在预训练阶段建立输入单元词典以及不明显的分割问题，该方法对于掩盖区域的预测损失、高质量的聚类步骤具有一定的鲁棒性，在多个基准测试集上的表现不低于当前最先进的 wav2vec 2.0 性能。

HuBERT：自监督语音表示学习通过隐藏单元的掩码预测

HuBERT: Self-Supervised Speech Representation Learning by Masked  Prediction of Hidden Units

Neural Machine Translation has achieved state-of-the-art performance for
several language pairs using a combination of parallel and synthetic data.
Synthetic data is often generated by back-translating sentences randomly
sampled from monolingual data using a reverse translation model. While
back-translation has been shown to be very effective in many cases, it is not
entirely clear why. In this work, we explore different aspects of
back-translation, and show that words with high prediction loss during training
benefit most from the addition of synthetic data. We introduce several
variations of sampling strategies targeting difficult-to-predict words using
prediction losses and frequencies of words. In addition, we also target the
contexts of difficult words and sample sentences that are similar in context.
Experimental results for the WMT news translation task show that our method
improves translation quality by up to 1.7 and 1.2 Bleu points over
back-translation using random sampling for German-English and English-German,
respectively.

该研究探讨了逆向翻译的不同方面，并表明在训练期间预测损失高的单词最能从合成数据的添加中获益。使用以困难预测单词的预测损失和单词频率的采样策略，以及类似背景的选取句子的策略比随机采样的逆向翻译方法在 WMT 新闻翻译任务中提高了翻译质量。在德语 - 英语和英语 - 德语方向上，翻译质量分别提高了 1.7 和 1.2 BLEU 分数。