Currently, reinforcement learning (RL), especially deep RL, has received more
and more attention in the research area. However, the security of RL has been
an obvious problem due to the attack manners becoming mature. In order to
defend against such adversarial attacks, several practical approaches are
developed, such as adversarial training, data filtering, etc. However, these
methods are mostly based on empirical algorithms and experiments, without
rigorous theoretical analysis of the robustness of the algorithms. In this
paper, we develop an algorithm to certify the robustness of a given policy
offline with random smoothing, which could be proven and conducted as
efficiently as ones without random smoothing. Experiments on different
environments confirm the correctness of our algorithm.

开发了一种算法，并使用随机平滑来获得对给定策略的离线鲁棒性认证，证明了其有效性，并在不同环境实验证明了算法的正确性。

鲁棒 离线强化学习 -- 确认置信区间

Robust Offline Reinforcement Learning -- Certify the Confidence Interval

Representational straightening refers to a decrease in curvature of visual
feature representations of a sequence of frames taken from natural movies.
Prior work established straightening in neural representations of the primate
primary visual cortex (V1) and perceptual straightening in human behavior as a
hallmark of biological vision in contrast to artificial feedforward neural
networks which did not demonstrate this phenomenon as they were not explicitly
optimized to produce temporally predictable movie representations. Here, we
show robustness to noise in the input image can produce representational
straightening in feedforward neural networks. Both adversarial training (AT)
and base classifiers for Random Smoothing (RS) induced remarkably straightened
feature codes. Demonstrating their utility within the domain of natural movies,
these codes could be inverted to generate intervening movie frames by linear
interpolation in the feature space even though they were not trained on these
trajectories. Demonstrating their biological utility, we found that AT and RS
training improved predictions of neural data in primate V1 over baseline models
providing a parsimonious, bio-plausible mechanism -- noise in the sensory input
stages -- for generating representations in early visual cortex. Finally, we
compared the geometric properties of frame representations in these networks to
better understand how they produced representations that mimicked the
straightening phenomenon from biology. Overall, this work elucidating emergent
properties of robust neural networks demonstrates that it is not necessary to
utilize predictive objectives or train directly on natural movie statistics to
achieve models supporting straightened movie representations similar to human
perception that also predict V1 neural responses.

鲁棒神经网络的出现性质证明，并非需要利用预测目标或直接训练于自然电影统计数据，便可以实现支持人类感知的影片表示，同时也能预测 V1 神经响应的模型。

鲁棒前馈神经网络中自然电影的类脑表示直线化

Brain-like representational straightening of natural movies in robust  feedforward neural networks

Random smoothing data augmentation is a unique form of regularization that
can prevent overfitting by introducing noise to the input data, encouraging the
model to learn more generalized features. Despite its success in various
applications, there has been a lack of systematic study on the regularization
ability of random smoothing. In this paper, we aim to bridge this gap by
presenting a framework for random smoothing regularization that can adaptively
and effectively learn a wide range of ground truth functions belonging to the
classical Sobolev spaces. Specifically, we investigate two underlying function
spaces: the Sobolev space of low intrinsic dimension, which includes the
Sobolev space in $D$-dimensional Euclidean space or low-dimensional
sub-manifolds as special cases, and the mixed smooth Sobolev space with a
tensor structure. By using random smoothing regularization as novel
convolution-based smoothing kernels, we can attain optimal convergence rates in
these cases using a kernel gradient descent algorithm, either with early
stopping or weight decay. It is noteworthy that our estimator can adapt to the
structural assumptions of the underlying data and avoid the curse of
dimensionality. This is achieved through various choices of injected noise
distributions such as Gaussian, Laplace, or general polynomial noises, allowing
for broad adaptation to the aforementioned structural assumptions of the
underlying data. The convergence rate depends only on the effective dimension,
which may be significantly smaller than the actual data dimension. We conduct
numerical experiments on simulated data to validate our theoretical results.

本文提出了随机平滑正则化的框架，通过使用卷积平滑核作为正则化方法，可以在 Sobolev 空间上最优收敛，并且可以适应底层数据的结构假设，从而避免维度灾难。