We propose Diffusion Noise Optimization (DNO), a new method that effectively
leverages existing motion diffusion models as motion priors for a wide range of
motion-related tasks. Instead of training a task-specific diffusion model for
each new task, DNO operates by optimizing the diffusion latent noise of an
existing pre-trained text-to-motion model. Given the corresponding latent noise
of a human motion, it propagates the gradient from the target criteria defined
on the motion space through the whole denoising process to update the diffusion
latent noise. As a result, DNO supports any use cases where criteria can be
defined as a function of motion. In particular, we show that, for motion
editing and control, DNO outperforms existing methods in both achieving the
objective and preserving the motion content. DNO accommodates a diverse range
of editing modes, including changing trajectory, pose, joint locations, or
avoiding newly added obstacles. In addition, DNO is effective in motion
denoising and completion, producing smooth and realistic motion from noisy and
partial inputs. DNO achieves these results at inference time without the need
for model retraining, offering great versatility for any defined reward or loss
function on the motion representation.

我们提出了扩散噪声优化（DNO），一种新的方法，通过有效利用现有的运动扩散模型作为广泛运动相关任务的运动先验，而无需为每个新任务训练特定于任务的扩散模型。DNO 支持任何可将准则定义为运动函数的用例，并在运动编辑和控制以及运动去噪和补全方面优于现有方法，同时使其具有适应多种编辑模式的能力，包括改变轨迹、姿势、关节位置或避开新添加的障碍物，并能在推理时间内实现这些结果而无需模型重训练，为运动表示的任何定义的奖励或损失函数提供了很大的灵活性。

优化扩散噪声可作为通用运动先验

Optimizing Diffusion Noise Can Serve As Universal Motion Priors

The progress in the generation of synthetic images has made it crucial to
assess their quality. While several metrics have been proposed to assess the
rendering of images, it is crucial for Text-to-Image (T2I) models, which
generate images based on a prompt, to consider additional aspects such as to
which extent the generated image matches the important content of the prompt.
Moreover, although the generated images usually result from a random starting
point, the influence of this one is generally not considered. In this article,
we propose a new metric based on prompt templates to study the alignment
between the content specified in the prompt and the corresponding generated
images. It allows us to better characterize the alignment in terms of the type
of the specified objects, their number, and their color. We conducted a study
on several recent T2I models about various aspects. An additional interesting
result we obtained with our approach is that image quality can vary drastically
depending on the latent noise used as a seed for the images. We also quantify
the influence of the number of concepts in the prompt, their order as well as
their (color) attributes. Finally, our method allows us to identify some latent
seeds that produce better images than others, opening novel directions of
research on this understudied topic.

通过针对 prompt 模板的新度量方法，本文对多种 Text-to-Image（T2I）模型的对齐性进行了研究，并发现潜在噪声和提示概念数量、顺序和属性等因素对图像生成的影响。

文本到图像生成对齐度评估度量 TIAM

TIAM -- A Metric for Evaluating Alignment in Text-to-Image Generation

Data-driven approaches to solving robotic tasks have gained a lot of traction
in recent years. However, most existing policies are trained on large-scale
datasets collected in curated lab settings. If we aim to deploy these models in
unstructured visual environments like people's homes, they will be unable to
cope with the mismatch in data distribution. In such light, we present the
first systematic effort in collecting a large dataset for robotic grasping in
homes. First, to scale and parallelize data collection, we built a low cost
mobile manipulator assembled for under 3K USD. Second, data collected using low
cost robots suffer from noisy labels due to imperfect execution and calibration
errors. To handle this, we develop a framework which factors out the noise as a
latent variable. Our model is trained on 28K grasps collected in several houses
under an array of different environmental conditions. We evaluate our models by
physically executing grasps on a collection of novel objects in multiple unseen
homes. The models trained with our home dataset showed a marked improvement of
43.7% over a baseline model trained with data collected in lab. Our
architecture which explicitly models the latent noise in the dataset also
performed 10% better than one that did not factor out the noise. We hope this
effort inspires the robotics community to look outside the lab and embrace
learning based approaches to handle inaccurate cheap robots.

研究表明，大多数基于数据驱动的机器人任务都是在实验室环境中收集的大规模数据集上进行训练的，该论文提出了在人们的家庭环境中使用低成本机器人手动搬运收集数据的方法，并开发了一种框架来处理数据中的噪音标签。该模型在采集了 28,000 次抓取数据并针对不同环境条件进行物理性能测试后，相较于实验室收集的数据，展现了明显的 43.7% 的改善，该模型还比其他模型表现 10% 更好。