Self-portraits captured from a short distance might look unnatural or even
unattractive due to heavy distortions making facial features malformed, and
ill-placed head poses. In this paper, we propose SUPER, a novel method of
eliminating distortions and adjusting head pose in a close-up face crop. We
perform 3D GAN inversion for a facial image by optimizing camera parameters and
face latent code, which gives a generated image. Besides, we estimate depth
from the obtained latent code, create a depth-induced 3D mesh, and render it
with updated camera parameters to obtain a warped portrait. Finally, we apply
the visibility-based blending so that visible regions are reprojected, and
occluded parts are restored with a generative model. Experiments on face
undistortion benchmarks and on our self-collected Head Rotation dataset (HeRo),
show that SUPER outperforms previous approaches both qualitatively and
quantitatively, opening new possibilities for photorealistic selfie editing.

本文提出了一种名为 SUPER 的新方法，用于消除自拍照片中的失真和调整头部姿势，实现了更真实的自拍照片编辑。通过 3D GAN 反向传播优化相机参数和面部隐藏编码，生成一张图像。此外，估计获得的隐藏编码的深度，创建深度感应的 3D 网格，并使用更新的相机参数进行渲染，获得一个形变的自画像。最后，通过基于可见性的融合，重新投影可见区域，并使用生成模型恢复被遮挡的部分。对自拍照片失真基准和自行收集的头部旋转数据集（HeRo）的实验表明，SUPER 在定性和定量上均优于以前的方法，为实现逼真的自拍照片编辑开辟了新的可能性。

SUPER：自拍矫正和头部角度编辑与身份保留

SUPER: Selfie Undistortion and Head Pose Editing with Identity  Preservation

Traditional control and task automation have been successfully demonstrated
in a variety of structured, controlled environments through the use of highly
specialized modeled robotic systems in conjunction with multiple sensors.
However, the application of autonomy in endoscopic surgery is very challenging,
particularly in soft tissue work, due to the lack of high-quality images and
the unpredictable, constantly deforming environment. In this work, we propose a
novel surgical perception framework, SuPer, for surgical robotic control. This
framework continuously collects 3D geometric information that allows for
mapping a deformable surgical field while tracking rigid instruments within the
field. To achieve this, a model-based tracker is employed to localize the
surgical tool with a kinematic prior in conjunction with a model-free tracker
to reconstruct the deformable environment and provide an estimated point cloud
as a mapping of the environment. The proposed framework was implemented on the
da Vinci Surgical System in real-time with an end-effector controller where the
target configurations are set and regulated through the framework. Our proposed
framework successfully completed soft tissue manipulation tasks with high
accuracy. The demonstration of this novel framework is promising for the future
of surgical autonomy. In addition, we provide our dataset for further surgical
research.

本研究提出了一种新颖的外科手术知觉框架 SuPer，该框架使用基于模型的追踪器与无模型追踪器相结合，实时地收集 3D 几何信息并构建可变形手术视野，成功地实现了对软组织的精确操控任务。同时，我们将提供本研究的数据集用于未来的外科手术研究。