The enduring inability of image generative models to recreate intricate
geometric features, such as those present in human hands and fingers has been
an ongoing problem in image generation for nearly a decade. While strides have
been made by increasing model sizes and diversifying training datasets, this
issue remains prevalent across all models, from denoising diffusion models to
Generative Adversarial Networks (GAN), pointing to a fundamental shortcoming in
the underlying architectures. In this paper, we demonstrate how this problem
can be mitigated by augmenting convolution layers geometric capabilities
through providing them with a single input channel incorporating the relative
$n$-dimensional Cartesian coordinate system. We show that this drastically
improves quality of hand and face images generated by GANs and Variational
AutoEncoders (VAE).

通过在卷积层中引入单一输入通道，加入与相对 $n$ 维笛卡尔坐标系相关的信息，我们展示了如何改进生成对抗网络（GAN）和变分自动编码器（VAE）生成的手部和面部图像的质量。

仅使用卷积是否能生成逼真的手部形象？

Can We Generate Realistic Hands Only Using Convolution?

Existing methods for 3D tracking from monocular RGB videos predominantly
consider articulated and rigid objects. Modelling dense non-rigid object
deformations in this setting remained largely unaddressed so far, although such
effects can improve the realism of the downstream applications such as AR/VR
and avatar communications. This is due to the severe ill-posedness of the
monocular view setting and the associated challenges. While it is possible to
naively track multiple non-rigid objects independently using 3D templates or
parametric 3D models, such an approach would suffer from multiple artefacts in
the resulting 3D estimates such as depth ambiguity, unnatural intra-object
collisions and missing or implausible deformations. Hence, this paper
introduces the first method that addresses the fundamental challenges depicted
above and that allows tracking human hands interacting with human faces in 3D
from single monocular RGB videos. We model hands as articulated objects
inducing non-rigid face deformations during an active interaction. Our method
relies on a new hand-face motion and interaction capture dataset with realistic
face deformations acquired with a markerless multi-view camera system. As a
pivotal step in its creation, we process the reconstructed raw 3D shapes with
position-based dynamics and an approach for non-uniform stiffness estimation of
the head tissues, which results in plausible annotations of the surface
deformations, hand-face contact regions and head-hand positions. At the core of
our neural approach are a variational auto-encoder supplying the hand-face
depth prior and modules that guide the 3D tracking by estimating the contacts
and the deformations. Our final 3D hand and face reconstructions are realistic
and more plausible compared to several baselines applicable in our setting,
both quantitatively and qualitatively.
this https URL

从单目 RGB 视频中追踪人手与人脸之间的 3D 交互，并建立了可靠的 3D 手部和面部重构方法。