Many deep reinforcement learning algorithms contain inductive biases that
sculpt the agent's objective and its interface to the environment. These
inductive biases can take many forms, including domain knowledge and pretuned
hyper-parameters. In general, there is a trade-off between generality and
performance when algorithms use such biases. Stronger biases can lead to faster
learning, but weaker biases can potentially lead to more general algorithms.
This trade-off is important because inductive biases are not free; substantial
effort may be required to obtain relevant domain knowledge or to tune
hyper-parameters effectively. In this paper, we re-examine several
domain-specific components that bias the objective and the environmental
interface of common deep reinforcement learning agents. We investigated whether
the performance deteriorates when these components are replaced with adaptive
solutions from the literature. In our experiments, performance sometimes
decreased with the adaptive components, as one might expect when comparing to
components crafted for the domain, but sometimes the adaptive components
performed better. We investigated the main benefit of having fewer
domain-specific components, by comparing the learning performance of the two
systems on a different set of continuous control problems, without additional
tuning of either system. As hypothesized, the system with adaptive components
performed better on many of the new tasks.

本文研究了深度强化学习算法中的归纳偏见及其对算法性能的影响，并通过实验发现了某些领域特定组件的更有效的自适应解决方案可能会提高算法性能。

关于深度强化学习中的归纳偏差

On Inductive Biases in Deep Reinforcement Learning

We study a basic private estimation problem: each of $n$ users draws a single
i.i.d. sample from an unknown Gaussian distribution, and the goal is to
estimate the mean of this Gaussian distribution while satisfying local
differential privacy for each user. Informally, local differential privacy
requires that each data point is individually and independently privatized
before it is passed to a learning algorithm. Locally private Gaussian
estimation is therefore difficult because the data domain is unbounded: users
may draw arbitrarily different inputs, but local differential privacy
nonetheless mandates that different users have (worst-case) similar privatized
output distributions. We provide both adaptive two-round solutions and
nonadaptive one-round solutions for locally private Gaussian estimation. We
then partially match these upper bounds with an information-theoretic lower
bound. This lower bound shows that our accuracy guarantees are tight up to
logarithmic factors for all sequentially interactive
$(\varepsilon,\delta)$-locally private protocols.

研究局部差分隐私在高斯分布参数估计中的应用，并给出了自适应的两轮解决方案和非自适应的一轮解决方案，并通过信息理论下界证明了准确性保证的紧密性。