Delivering immersive, 3D experiences for human communication requires a
method to obtain 360 degree photo-realistic avatars of humans. To make these
experiences accessible to all, only commodity hardware, like mobile phone
cameras, should be necessary to capture the data needed for avatar creation.
For avatars to be rendered realistically from any viewpoint, we require
training images and camera poses from all angles. However, we cannot rely on
there being trackable features in the foreground or background of all images
for use in estimating poses, especially from the side or back of the head. To
overcome this, we propose a novel landmark detector trained on synthetic data
to estimate camera poses from 360 degree mobile phone videos of a human head
for use in a multi-stage optimization process which creates a photo-realistic
avatar. We perform validation experiments with synthetic data and showcase our
method on 360 degree avatars trained from mobile phone videos.

为了实现人类交流的沉浸式 3D 体验，需使用像手机摄像头一样的普通硬件获取 360 度的逼真人形头像，并使用经过训练的关键点检测器和多阶段优化过程创建头像。

野外环境下的全景真实头像模拟

Photo-realistic 360 Head Avatars in the Wild

Retrieving object instances among cluttered scenes efficiently requires
compact yet comprehensive regional image representations. Intuitively, object
semantics can help build the index that focuses on the most relevant regions.
However, due to the lack of bounding-box datasets for objects of interest among
retrieval benchmarks, most recent work on regional representations has focused
on either uniform or class-agnostic region selection. In this paper, we first
fill the void by providing a new dataset of landmark bounding boxes, based on
the Google Landmarks dataset, that includes $86k$ images with manually curated
boxes from $15k$ unique landmarks. Then, we demonstrate how a trained landmark
detector, using our new dataset, can be leveraged to index image regions and
improve retrieval accuracy while being much more efficient than existing
regional methods. In addition, we introduce a novel regional aggregated
selective match kernel (R-ASMK) to effectively combine information from
detected regions into an improved holistic image representation. R-ASMK boosts
image retrieval accuracy substantially with no dimensionality increase, while
even outperforming systems that index image regions independently. Our complete
image retrieval system improves upon the previous state-of-the-art by
significant margins on the Revisited Oxford and Paris datasets. Code and data
available at the project webpage:
this https URL

本文通过提供新的地标边界框数据集，展示了如何使用训练良好的地标探测器，在基于地标的检索系统中更高效地获取图像区域，并引入了一种新的区域聚合选择匹配核 (R-ASMK) 来提高图像检索精度。该图像检索系统在 Revisited Oxford 和 Paris 数据集上显著超越了之前的最新技术。