Existing imitation learning (IL) methods such as inverse reinforcement
learning (IRL) usually have a double-loop training process, alternating between
learning a reward function and a policy and tend to suffer long training time
and high variance. In this work, we identify the benefits of differentiable
physics simulators and propose a new IL method, i.e., Imitation Learning via
Differentiable Physics (ILD), which gets rid of the double-loop design and
achieves significant improvements in final performance, convergence speed, and
stability. The proposed ILD incorporates the differentiable physics simulator
as a physics prior into its computational graph for policy learning. It unrolls
the dynamics by sampling actions from a parameterized policy, simply minimizing
the distance between the expert trajectory and the agent trajectory, and
back-propagating the gradient into the policy via temporal physics operators.
With the physics prior, ILD policies can not only be transferable to unseen
environment specifications but also yield higher final performance on a variety
of tasks. In addition, ILD naturally forms a single-loop structure, which
significantly improves the stability and training speed. To simplify the
complex optimization landscape induced by temporal physics operations, ILD
dynamically selects the learning objectives for each state during optimization.
In our experiments, we show that ILD outperforms state-of-the-art methods in a
variety of continuous control tasks with Brax, requiring only one expert
demonstration. In addition, ILD can be applied to challenging deformable object
manipulation tasks and can be generalized to unseen configurations.

本文提出了一种新的基于可微分物理仿真器的模仿学习方法 (ILD)，该方法将物理预设作为先验加入到计算图中进行策略学习，在优化过程中动态选择每个状态的学习目标，实现了单循环结构，提高了稳定性和训练速度。在验证中，ILD 在连续控制任务和变形物体操作任务中表现优异，且只需要一次专家演示。

可微分物理学的模仿学习

Imitation Learning via Differentiable Physics

We consider the problem of sequential robotic manipulation of deformable
objects using tools. Previous works have shown that differentiable physics
simulators provide gradients to the environment state and help trajectory
optimization to converge orders of magnitude faster than model-free
reinforcement learning algorithms for deformable object manipulation. However,
such gradient-based trajectory optimization typically requires access to the
full simulator states and can only solve short-horizon, single-skill tasks due
to local optima. In this work, we propose a novel framework, named DiffSkill,
that uses a differentiable physics simulator for skill abstraction to solve
long-horizon deformable object manipulation tasks from sensory observations. In
particular, we first obtain short-horizon skills using individual tools from a
gradient-based optimizer, using the full state information in a differentiable
simulator; we then learn a neural skill abstractor from the demonstration
trajectories which takes RGBD images as input. Finally, we plan over the skills
by finding the intermediate goals and then solve long-horizon tasks. We show
the advantages of our method in a new set of sequential deformable object
manipulation tasks compared to previous reinforcement learning algorithms and
compared to the trajectory optimizer.

本研究提出了一个名为 DiffSkill 的新框架，使用可微分的物理模拟器进行技能抽象，从感官观测中解决长期目标可达的可变形物体操作任务。

DiffSkill: 从可微分物理学中抽象技能用于工具对形变物体的操作

DiffSkill: Skill Abstraction from Differentiable Physics for Deformable  Object Manipulations with Tools

Shape-from-Template (SfT) methods estimate 3D surface deformations from a
single monocular RGB camera while assuming a 3D state known in advance (a
template). This is an important yet challenging problem due to the
under-constrained nature of the monocular setting. Existing SfT techniques
predominantly use geometric and simplified deformation models, which often
limits their reconstruction abilities. In contrast to previous works, this
paper proposes a new SfT approach explaining 2D observations through physical
simulations accounting for forces and material properties. Our differentiable
physics simulator regularises the surface evolution and optimises the material
elastic properties such as bending coefficients, stretching stiffness and
density. We use a differentiable renderer to minimise the dense reprojection
error between the estimated 3D states and the input images and recover the
deformation parameters using an adaptive gradient-based optimisation. For the
evaluation, we record with an RGB-D camera challenging real surfaces exposed to
physical forces with various material properties and textures. Our approach
significantly reduces the 3D reconstruction error compared to multiple
competing methods. For the source code and data, see
this https URL

该研究提出了一种使用可微物理模拟方法的 SfT 方法，该方法具有更高的重建精度，并通过 RGB-D 相机采集数据进行验证。

基于物理模型的模板形态恢复 (φ-SfT)

φ-SfT: Shape-from-Template with a Physics-Based Deformation Model

The incorporation of appropriate inductive bias plays a critical role in
learning dynamics from data. A growing body of work has been exploring ways to
enforce energy conservation in the learned dynamics by encoding Lagrangian or
Hamiltonian dynamics into the neural network architecture. These existing
approaches are based on differential equations, which do not allow
discontinuity in the states and thereby limit the class of systems one can
learn. However, in reality, most physical systems, such as legged robots and
robotic manipulators, involve contacts and collisions, which introduce
discontinuities in the states. In this paper, we introduce a differentiable
contact model, which can capture contact mechanics: frictionless/frictional, as
well as elastic/inelastic. This model can also accommodate inequality
constraints, such as limits on the joint angles. The proposed contact model
extends the scope of Lagrangian and Hamiltonian neural networks by allowing
simultaneous learning of contact and system properties. We demonstrate this
framework on a series of challenging 2D and 3D physical systems with different
coefficients of restitution and friction. The learned dynamics can be used as a
differentiable physics simulator for downstream gradient-based optimization
tasks, such as planning and control.

本文提出一种可微分的接触模型，扩展了拉格朗日和哈密敦神经网络的范围，并演示了在各种不同的物理系统上，该学习动态可以用作下游基于梯度的优化任务（如计划和控制）的不同 iable 物理模拟器。