If $A$ and $B$ are sets such that $A \subset B$, generalisation may be
understood as the inference from $A$ of a hypothesis sufficient to construct
$B$. One might infer any number of hypotheses from $A$, yet only some of those
may generalise to $B$. How can one know which are likely to generalise? One
strategy is to choose the shortest, equating the ability to compress
information with the ability to generalise (a proxy for intelligence). We
examine this in the context of a mathematical formalism of enactive cognition.
We show that compression is neither necessary nor sufficient to maximise
performance (measured in terms of the probability of a hypothesis
generalising). We formulate a proxy unrelated to length or simplicity, called
weakness. We show that if tasks are uniformly distributed, then there is no
choice of proxy that performs at least as well as weakness maximisation in all
tasks while performing strictly better in at least one. In other words,
weakness is the pareto optimal choice of proxy. In experiments comparing
maximum weakness and minimum description length in the context of binary
arithmetic, the former generalised at between $1.1$ and $5$ times the rate of
the latter. We argue this demonstrates that weakness is a far better proxy, and
explains why Deepmind's Apperception Engine is able to generalise effectively.

本文探讨了压缩信息与推理方法，选择代理策略作为推理能力的代表，提出了弱点策略来最大化推理性能，并通过二进制算术的实验比较了最大弱点和最小描述长度。

假设的最佳选择是最弱的，而不是最短的

The Optimal Choice of Hypothesis Is the Weakest, Not the Shortest

Unsupervised domain adaptation (UDA) aims to train a target classifier with
labeled samples from the source domain and unlabeled samples from the target
domain. Classical UDA learning bounds show that target risk is upper bounded by
three terms: source risk, distribution discrepancy, and combined risk. Based on
the assumption that the combined risk is a small fixed value, methods based on
this bound train a target classifier by only minimizing estimators of the
source risk and the distribution discrepancy. However, the combined risk may
increase when minimizing both estimators, which makes the target risk
uncontrollable. Hence the target classifier cannot achieve ideal performance if
we fail to control the combined risk. To control the combined risk, the key
challenge takes root in the unavailability of the labeled samples in the target
domain. To address this key challenge, we propose a method named E-MixNet.
E-MixNet employs enhanced mixup, a generic vicinal distribution, on the labeled
source samples and pseudo-labeled target samples to calculate a proxy of the
combined risk. Experiments show that the proxy can effectively curb the
increase of the combined risk when minimizing the source risk and distribution
discrepancy. Furthermore, we show that if the proxy of the combined risk is
added into loss functions of four representative UDA methods, their performance
is also improved.

提出了一种名为 E-MixNet 的方法，旨在通过新的混合技术来评估目标域的估计风险，并在目标域缺乏标记样本的情况下来控制组合风险，从而提高无监督领域适应的性能。