Understanding classifier decision under novel environments is central to the
community, and a common practice is evaluating it on labeled test sets.
However, in real-world testing, image annotations are difficult and expensive
to obtain, especially when the test environment is changing. A natural question
then arises: given a trained classifier, can we evaluate its accuracy on
varying unlabeled test sets? In this work, we train semantic classification and
rotation prediction in a multi-task way. On a series of datasets, we report an
interesting finding, i.e., the semantic classification accuracy exhibits a
strong linear relationship with the accuracy of the rotation prediction task
(Pearson's Correlation r > 0.88). This finding allows us to utilize linear
regression to estimate classifier performance from the accuracy of rotation
prediction which can be obtained on the test set through the freely generated
rotation labels.

本文针对图像分类器在新环境下的分类决策问题，通过多任务学习方法，发现语义分类精度与旋转预测任务准确性有强线性关系（Pearson 相关系数 r> 0.88），从旋转预测任务的准确性，应用线性回归估计分类器性能。

旋转预测对分类器在不同测试环境下准确性的影响

What Does Rotation Prediction Tell Us about Classifier Accuracy under  Varying Testing Environments?

We propose the challenge of rapid task-solving in novel environments (RTS),
wherein an agent must solve a series of tasks as rapidly as possible in an
unfamiliar environment. An effective RTS agent must balance between exploring
the unfamiliar environment and solving its current task, all while building a
model of the new environment over which it can plan when faced with later
tasks. While modern deep RL agents exhibit some of these abilities in
isolation, none are suitable for the full RTS challenge. To enable progress
toward RTS, we introduce two challenge domains: (1) a minimal RTS challenge
called the Memory&Planning Game and (2) One-Shot StreetLearn Navigation, which
introduces scale and complexity from real-world data. We demonstrate that
state-of-the-art deep RL agents fail at RTS in both domains, and that this
failure is due to an inability to plan over gathered knowledge. We develop
Episodic Planning Networks (EPNs) and show that deep-RL agents with EPNs excel
at RTS, outperforming the nearest baseline by factors of 2-3 and learning to
navigate held-out StreetLearn maps within a single episode. We show that EPNs
learn to execute a value iteration-like planning algorithm and that they
generalize to situations beyond their training experience. algorithm and that
they generalize to situations beyond their training experience.

在未知环境中提出了快速任务解决的挑战，引入了两个具有挑战性的任务领域进行研究，发现深度强化学习算法无法在其中表现出色，提出了一种名为 EPN 的新模型优于深度强化学习算法.

新环境下的快速任务解决

Rapid Task-Solving in Novel Environments

Model-based control is a popular paradigm for robot navigation because it can
leverage a known dynamics model to efficiently plan robust robot trajectories.
However, it is challenging to use model-based methods in settings where the
environment is a priori unknown and can only be observed partially through
on-board sensors on the robot. In this work, we address this short-coming by
coupling model-based control with learning-based perception. The learning-based
perception module produces a series of waypoints that guide the robot to the
goal via a collision-free path. These waypoints are used by a model-based
planner to generate a smooth and dynamically feasible trajectory that is
executed on the physical system using feedback control. Our experiments in
simulated real-world cluttered environments and on an actual ground vehicle
demonstrate that the proposed approach can reach goal locations more reliably
and efficiently in novel environments as compared to purely geometric
mapping-based or end-to-end learning-based alternatives. Our approach does not
rely on detailed explicit 3D maps of the environment, works well with low frame
rates, and generalizes well from simulation to the real world. Videos
describing our approach and experiments are available on the project website.

通过将模型驱动控制与基于学习的感知相结合，本研究提出了一种适用于未知场景下机器人导航的方法，实验结果表明，相比于几何映射和终端学习方法，该方法在处理复杂环境下的目标到达更加可靠和高效。本方法不依赖于对环境的详细、明确的 3D 地图，适用于低帧率，且具有良好的仿真到真实世界的泛化性能。