In the absence of parallax cues, a learning-based single image depth
estimation (SIDE) model relies heavily on shading and contextual cues in the
image. While this simplicity is attractive, it is necessary to train such
models on large and varied datasets, which are difficult to capture. It has
been shown that using embeddings from pre-trained foundational models, such as
CLIP, improves zero shot transfer in several applications. Taking inspiration
from this, in our paper we explore the use of global image priors generated
from a pre-trained ViT model to provide more detailed contextual information.
We argue that the embedding vector from a ViT model, pre-trained on a large
dataset, captures greater relevant information for SIDE than the usual route of
generating pseudo image captions, followed by CLIP based text embeddings. Based
on this idea, we propose a new SIDE model using a diffusion backbone which is
conditioned on ViT embeddings. Our proposed design establishes a new
state-of-the-art (SOTA) for SIDE on NYUv2 dataset, achieving Abs Rel error of
0.059(14% improvement) compared to 0.069 by the current SOTA (VPD). And on
KITTI dataset, achieving Sq Rel error of 0.139 (2% improvement) compared to
0.142 by the current SOTA (GEDepth). For zero-shot transfer with a model
trained on NYUv2, we report mean relative improvement of (20%, 23%, 81%, 25%)
over NeWCRFs on (Sun-RGBD, iBims1, DIODE, HyperSim) datasets, compared to (16%,
18%, 45%, 9%) by ZoeDepth. The code is available at
this https URL

使用预训练的 ViT 模型的全局图像先验信息为单图深度估计提供更详细的上下文信息，从而建立了新的领先优势的模型，并在多个数据集上实现了显著的改进。

ECoDepth: 单目深度估计的有效扩展方法

ECoDepth: Effective Conditioning of Diffusion Models for Monocular Depth  Estimation

Living creatures and machines interact with the world through their
morphology and motions. Recent advances in creating bio-inspired morphing
robots and machines have led to the study of variable geometry truss (VGT),
structures that can approximate arbitrary geometries and has large degree of
freedom to deform. However, they are limited to simple geometries and motions
due to the excessively complex control system. While a recent work PneuMesh
solves this challenge with a novel VGT design that introduces a selective
channel connection strategy, it imposes new challenge in identifying effective
channel groupings and control methods.
Building on top of the hardware concept presented in PneuMesh, we frame the
challenge into a co-design problem and introduce a learning-based model to find
a sub-optimal design. Specifically, given an initial truss structure provided
by a human designer, we first adopt a genetic algorithm (GA) to optimize the
channel grouping, and then couple GA with reinforcement learning (RL) for the
control. The model is tailored to the PneuMesh system with customized
initialization, mutation and selection functions, as well as the customized
translation-invariant state vector for reinforcement learning. The result shows
that our method enables a robotic table-based VGT to achieve various motions
with a limited number of control inputs. The table is trained to move, lower
its body or tilt its tabletop to accommodate multiple use cases such as
benefiting kids and painters to use it in different shape states, allowing
inclusive and adaptive design through morphing trusses.

本研究基于 PneuMesh 硬件构想，设计了一个学习模型和遗传算法和强化学习相结合的优化方案，以帮助机械桌在少数控制输入的情况下实现各种运动，包括变形桁架的包容性和适应性设计。

可变形桁架的计算机协同设计

Computational Co-Design for Variable Geometry Truss

One of the key challenges in high speed off road navigation on ground
vehicles is that the kinodynamics of the vehicle terrain interaction can differ
dramatically depending on the terrain. Previous approaches to addressing this
challenge have considered learning an inverse kinodynamics (IKD) model,
conditioned on inertial information of the vehicle to sense the kinodynamic
interactions. In this paper, we hypothesize that to enable accurate high-speed
off-road navigation using a learned IKD model, in addition to inertial
information from the past, one must also anticipate the kinodynamic
interactions of the vehicle with the terrain in the future. To this end, we
introduce Visual-Inertial Inverse Kinodynamics (VI-IKD), a novel learning based
IKD model that is conditioned on visual information from a terrain patch ahead
of the robot in addition to past inertial information, enabling it to
anticipate kinodynamic interactions in the future. We validate the
effectiveness of VI-IKD in accurate high-speed off-road navigation
experimentally on a scale 1/5 UT-AlphaTruck off-road autonomous vehicle in both
indoor and outdoor environments and show that compared to other
state-of-the-art approaches, VI-IKD enables more accurate and robust off-road
navigation on a variety of different terrains at speeds of up to 3.5 m/s.

该论文提出了一种新的基于学习的运动学逆解模型：Visual-Inertial Inverse Kinodynamics (VI-IKD)，它结合了惯性和视觉信息，可以预测地形与车辆间的运动学相互作用，提高了无人车高速越野导航的准确性和鲁棒性。

VI-IKD：利用学习的视觉惯性反运动学进行高速准确的越野导航

VI-IKD: High-Speed Accurate Off-Road Navigation using Learned Visual-Inertial Inverse Kinodynamics

Environment perception, including object detection and distance estimation,
is one of the most crucial tasks for autonomous driving. Many attentions have
been paid on the object detection task, but distance estimation only arouse few
interests in the computer vision community. Observing that the traditional
inverse perspective mapping algorithm performs poorly for objects far away from
the camera or on the curved road, in this paper, we address the challenging
distance estimation problem by developing the first end-to-end learning-based
model to directly predict distances for given objects in the images. Besides
the introduction of a learning-based base model, we further design an enhanced
model with a keypoint regressor, where a projection loss is defined to enforce
a better distance estimation, especially for objects close to the camera. To
facilitate the research on this task, we construct the extented KITTI and
nuScenes (mini) object detection datasets with a distance for each object. Our
experiments demonstrate that our proposed methods outperform alternative
approaches (e.g., the traditional IPM, SVR) on object-specific distance
estimation, particularly for the challenging cases that objects are on a curved
road. Moreover, the performance margin implies the effectiveness of our
enhanced method.

本文提出了一种基于学习的方法，通过直接预测图像中给定物体的距离来解决自动驾驶中一个困难的问题：距离估计。该方法在构建了 KITTI 和 nuScenes 数据集的基础上，利用增强模型和关键点回归器解决了传统的 IPM、SVR 方法不能很好解决的场景，取得了较大的性能提升。